Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 5,
 'device': device(type='cuda', index=0),
 'dropout': 0.2,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1200,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'grad_clip': None,
 'hidden_size': 1200,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.706573
Epoch 1
Validation auc = 0.730986
Epoch 2
Validation auc = 0.706573
Epoch 3
Validation auc = 0.755869
Epoch 4
Loss = 5.3973e-01, PNorm = 63.7740, GNorm = 3.2912, lr_0 = 6.5000e-04
Validation auc = 0.736150
Epoch 5
Validation auc = 0.750235
Epoch 6
Validation auc = 0.747887
Epoch 7
Validation auc = 0.743662
Epoch 8
Validation auc = 0.735681
Epoch 9
Loss = 3.8519e-01, PNorm = 63.9385, GNorm = 2.4332, lr_0 = 9.9743e-04
Validation auc = 0.734742
Epoch 10
Validation auc = 0.727700
Epoch 11
Validation auc = 0.714085
Epoch 12
Validation auc = 0.717371
Epoch 13
Validation auc = 0.725352
Epoch 14
Loss = 2.8294e-01, PNorm = 64.1178, GNorm = 2.3597, lr_0 = 9.8890e-04
Validation auc = 0.733333
Epoch 15
Validation auc = 0.736150
Epoch 16
Validation auc = 0.729108
Epoch 17
Validation auc = 0.739437
Epoch 18
Validation auc = 0.743662
Epoch 19
Loss = 2.3469e-01, PNorm = 64.2491, GNorm = 1.4789, lr_0 = 9.8045e-04
Validation auc = 0.745070
Epoch 20
Validation auc = 0.741784
Epoch 21
Validation auc = 0.736150
Epoch 22
Validation auc = 0.739437
Epoch 23
Validation auc = 0.730047
Epoch 24
Loss = 2.6271e-01, PNorm = 64.3496, GNorm = 1.1085, lr_0 = 9.7207e-04
Validation auc = 0.725352
Epoch 25
Validation auc = 0.729108
Epoch 26
Validation auc = 0.733333
Epoch 27
Validation auc = 0.733333
Epoch 28
Validation auc = 0.727700
Epoch 29
Loss = 1.5639e-01, PNorm = 64.4422, GNorm = 1.4499, lr_0 = 9.6376e-04
Validation auc = 0.718310
Epoch 30
Validation auc = 0.717840
Epoch 31
Validation auc = 0.729577
Epoch 32
Validation auc = 0.726291
Epoch 33
Validation auc = 0.701878
Epoch 34
Loss = 1.6016e-01, PNorm = 64.5410, GNorm = 1.1528, lr_0 = 9.5552e-04
Validation auc = 0.697183
Epoch 35
Validation auc = 0.708451
Epoch 36
Validation auc = 0.722535
Epoch 37
Validation auc = 0.725352
Epoch 38
Validation auc = 0.722066
Epoch 39
Loss = 1.5581e-01, PNorm = 64.6543, GNorm = 1.4199, lr_0 = 9.4735e-04
Validation auc = 0.711737
Epoch 40
Validation auc = 0.722535
Epoch 41
Validation auc = 0.740845
Epoch 42
Validation auc = 0.744601
Epoch 43
Validation auc = 0.746009
Epoch 44
Loss = 9.6010e-02, PNorm = 64.7662, GNorm = 0.8253, lr_0 = 9.3925e-04
Validation auc = 0.736150
Epoch 45
Validation auc = 0.735211
Epoch 46
Validation auc = 0.737089
Epoch 47
Validation auc = 0.750235
Epoch 48
Validation auc = 0.753521
Epoch 49
Loss = 1.2325e-01, PNorm = 64.8779, GNorm = 1.2542, lr_0 = 9.3122e-04
Validation auc = 0.751643
Epoch 50
Validation auc = 0.746948
Epoch 51
Validation auc = 0.736620
Epoch 52
Validation auc = 0.735211
Epoch 53
Validation auc = 0.735681
Epoch 54
Loss = 1.6036e-01, PNorm = 65.0131, GNorm = 1.1208, lr_0 = 9.2326e-04
Validation auc = 0.732394
Epoch 55
Validation auc = 0.726761
Epoch 56
Validation auc = 0.728169
Epoch 57
Validation auc = 0.725822
Epoch 58
Validation auc = 0.720657
Epoch 59
Loss = 1.3117e-01, PNorm = 65.1712, GNorm = 1.7449, lr_0 = 9.1537e-04
Validation auc = 0.720657
Epoch 60
Validation auc = 0.730986
Epoch 61
Validation auc = 0.729108
Epoch 62
Validation auc = 0.728638
Epoch 63
Validation auc = 0.733333
Epoch 64
Loss = 6.9837e-02, PNorm = 65.3258, GNorm = 0.9804, lr_0 = 9.0754e-04
Validation auc = 0.731455
Epoch 65
Validation auc = 0.728638
Epoch 66
Validation auc = 0.730986
Epoch 67
Validation auc = 0.726761
Epoch 68
Validation auc = 0.726761
Epoch 69
Loss = 1.0372e-01, PNorm = 65.4747, GNorm = 0.4991, lr_0 = 8.9978e-04
Validation auc = 0.722066
Epoch 70
Validation auc = 0.724883
Epoch 71
Validation auc = 0.731925
Epoch 72
Validation auc = 0.728638
Epoch 73
Validation auc = 0.723474
Epoch 74
Loss = 9.6365e-02, PNorm = 65.6197, GNorm = 0.5319, lr_0 = 8.9209e-04
Validation auc = 0.719718
Epoch 75
Validation auc = 0.715023
Epoch 76
Validation auc = 0.722066
Epoch 77
Validation auc = 0.728169
Epoch 78
Validation auc = 0.727700
Epoch 79
Loss = 7.2588e-02, PNorm = 65.7579, GNorm = 0.6622, lr_0 = 8.8447e-04
Validation auc = 0.723944
Epoch 80
Validation auc = 0.727230
Epoch 81
Validation auc = 0.730047
Epoch 82
Validation auc = 0.731455
Epoch 83
Validation auc = 0.728169
Epoch 84
Loss = 5.6150e-02, PNorm = 65.9090, GNorm = 0.5558, lr_0 = 8.7691e-04
Validation auc = 0.728169
Epoch 85
Validation auc = 0.723944
Epoch 86
Validation auc = 0.720657
Epoch 87
Validation auc = 0.714085
Epoch 88
Validation auc = 0.711737
Epoch 89
Loss = 1.0295e-01, PNorm = 66.0344, GNorm = 0.9298, lr_0 = 8.6941e-04
Validation auc = 0.707981
Epoch 90
Validation auc = 0.707512
Epoch 91
Validation auc = 0.702347
Epoch 92
Validation auc = 0.707512
Epoch 93
Validation auc = 0.706103
Epoch 94
Loss = 9.6324e-02, PNorm = 66.1852, GNorm = 0.7790, lr_0 = 8.6198e-04
Validation auc = 0.714085
Epoch 95
Validation auc = 0.724413
Epoch 96
Validation auc = 0.727230
Epoch 97
Validation auc = 0.735681
Epoch 98
Validation auc = 0.732394
Epoch 99
Loss = 7.0159e-02, PNorm = 66.3137, GNorm = 0.6268, lr_0 = 8.5461e-04
Validation auc = 0.723944
Epoch 100
Validation auc = 0.720188
Epoch 101
Validation auc = 0.726291
Epoch 102
Validation auc = 0.735211
Epoch 103
Validation auc = 0.734272
Epoch 104
Loss = 6.0915e-02, PNorm = 66.4683, GNorm = 1.1748, lr_0 = 8.4730e-04
Validation auc = 0.723474
Epoch 105
Validation auc = 0.711737
Epoch 106
Validation auc = 0.709390
Epoch 107
Validation auc = 0.710798
Epoch 108
Validation auc = 0.716432
Epoch 109
Loss = 4.0478e-02, PNorm = 66.6270, GNorm = 0.5083, lr_0 = 8.4006e-04
Validation auc = 0.716901
Epoch 110
Validation auc = 0.717840
Epoch 111
Validation auc = 0.719718
Epoch 112
Validation auc = 0.718779
Epoch 113
Validation auc = 0.715023
Epoch 114
Loss = 7.0984e-02, PNorm = 66.7938, GNorm = 1.1452, lr_0 = 8.3288e-04
Validation auc = 0.712676
Epoch 115
Validation auc = 0.708920
Epoch 116
Validation auc = 0.713615
Epoch 117
Validation auc = 0.716432
Epoch 118
Validation auc = 0.714085
Epoch 119
Loss = 1.9356e-02, PNorm = 66.9137, GNorm = 0.2689, lr_0 = 8.2576e-04
Validation auc = 0.713146
Epoch 120
Validation auc = 0.717840
Epoch 121
Validation auc = 0.727230
Epoch 122
Validation auc = 0.736620
Epoch 123
Validation auc = 0.740845
Epoch 124
Loss = 8.0026e-02, PNorm = 67.0681, GNorm = 1.4301, lr_0 = 8.1870e-04
Validation auc = 0.735211
Epoch 125
Validation auc = 0.729108
Epoch 126
Validation auc = 0.723944
Epoch 127
Validation auc = 0.713146
Epoch 128
Validation auc = 0.712207
Epoch 129
Loss = 4.3860e-02, PNorm = 67.2205, GNorm = 0.4826, lr_0 = 8.1170e-04
Validation auc = 0.715962
Epoch 130
Validation auc = 0.712207
Epoch 131
Validation auc = 0.716901
Epoch 132
Validation auc = 0.715962
Epoch 133
Validation auc = 0.721127
Epoch 134
Loss = 4.6489e-02, PNorm = 67.3332, GNorm = 0.8802, lr_0 = 8.0476e-04
Validation auc = 0.721596
Epoch 135
Validation auc = 0.716901
Epoch 136
Validation auc = 0.721127
Epoch 137
Validation auc = 0.723474
Epoch 138
Validation auc = 0.728169
Epoch 139
Loss = 3.9996e-02, PNorm = 67.4300, GNorm = 0.9838, lr_0 = 7.9788e-04
Validation auc = 0.727700
Epoch 140
Validation auc = 0.723005
Epoch 141
Validation auc = 0.719718
Epoch 142
Validation auc = 0.722066
Epoch 143
Validation auc = 0.725822
Epoch 144
Loss = 3.0058e-02, PNorm = 67.5287, GNorm = 0.6784, lr_0 = 7.9106e-04
Validation auc = 0.721127
Epoch 145
Validation auc = 0.714085
Epoch 146
Validation auc = 0.711737
Epoch 147
Validation auc = 0.705634
Epoch 148
Validation auc = 0.705634
Epoch 149
Loss = 1.7716e-02, PNorm = 67.6228, GNorm = 0.2877, lr_0 = 7.8430e-04
Validation auc = 0.705634
Epoch 150
Validation auc = 0.711268
Epoch 151
Validation auc = 0.716432
Epoch 152
Validation auc = 0.722535
Epoch 153
Validation auc = 0.722066
Epoch 154
Loss = 1.5074e-02, PNorm = 67.7121, GNorm = 0.2027, lr_0 = 7.7759e-04
Validation auc = 0.721127
Epoch 155
Validation auc = 0.721596
Epoch 156
Validation auc = 0.719249
Epoch 157
Validation auc = 0.716901
Epoch 158
Validation auc = 0.717840
Epoch 159
Loss = 1.5573e-02, PNorm = 67.8033, GNorm = 0.4990, lr_0 = 7.7095e-04
Validation auc = 0.716901
Epoch 160
Validation auc = 0.714085
Epoch 161
Validation auc = 0.709859
Epoch 162
Validation auc = 0.711737
Epoch 163
Validation auc = 0.713615
Epoch 164
Loss = 7.8984e-02, PNorm = 67.8993, GNorm = 1.1108, lr_0 = 7.6436e-04
Validation auc = 0.718779
Epoch 165
Validation auc = 0.723944
Epoch 166
Validation auc = 0.722535
Epoch 167
Validation auc = 0.722535
Epoch 168
Validation auc = 0.721127
Epoch 169
Loss = 8.8599e-03, PNorm = 67.9947, GNorm = 0.1055, lr_0 = 7.5782e-04
Validation auc = 0.719718
Epoch 170
Validation auc = 0.718310
Epoch 171
Validation auc = 0.715023
Epoch 172
Validation auc = 0.715962
Epoch 173
Validation auc = 0.719249
Epoch 174
Loss = 2.4496e-02, PNorm = 68.1016, GNorm = 0.0844, lr_0 = 7.5134e-04
Validation auc = 0.718310
Epoch 175
Validation auc = 0.738028
Epoch 176
Validation auc = 0.737559
Epoch 177
Validation auc = 0.710329
Epoch 178
Validation auc = 0.707042
Epoch 179
Loss = 2.0708e-02, PNorm = 68.2163, GNorm = 0.4297, lr_0 = 7.4492e-04
Validation auc = 0.706103
Epoch 180
Validation auc = 0.705634
Epoch 181
Validation auc = 0.707042
Epoch 182
Validation auc = 0.705164
Epoch 183
Validation auc = 0.700469
Epoch 184
Loss = 4.4995e-02, PNorm = 68.3438, GNorm = 0.7961, lr_0 = 7.3855e-04
Validation auc = 0.709859
Epoch 185
Validation auc = 0.712207
Epoch 186
Validation auc = 0.714085
Epoch 187
Validation auc = 0.716901
Epoch 188
Validation auc = 0.722535
Epoch 189
Loss = 2.3834e-02, PNorm = 68.4815, GNorm = 0.7408, lr_0 = 7.3224e-04
Validation auc = 0.724413
Epoch 190
Validation auc = 0.722066
Epoch 191
Validation auc = 0.720657
Epoch 192
Validation auc = 0.717840
Epoch 193
Validation auc = 0.719249
Epoch 194
Loss = 1.5744e-02, PNorm = 68.5966, GNorm = 0.6356, lr_0 = 7.2598e-04
Validation auc = 0.719249
Epoch 195
Validation auc = 0.718779
Epoch 196
Validation auc = 0.720657
Epoch 197
Validation auc = 0.718779
Epoch 198
Validation auc = 0.719718
Epoch 199
Loss = 1.3425e-02, PNorm = 68.6904, GNorm = 0.3178, lr_0 = 7.1977e-04
Validation auc = 0.721127
Epoch 200
Validation auc = 0.716432
Epoch 201
Validation auc = 0.713615
Epoch 202
Validation auc = 0.708920
Epoch 203
Validation auc = 0.709859
Epoch 204
Loss = 2.6067e-02, PNorm = 68.7857, GNorm = 0.4890, lr_0 = 7.1362e-04
Validation auc = 0.715493
Epoch 205
Validation auc = 0.719718
Epoch 206
Validation auc = 0.724883
Epoch 207
Validation auc = 0.727230
Epoch 208
Validation auc = 0.728169
Epoch 209
Loss = 1.0516e-02, PNorm = 68.8874, GNorm = 0.4160, lr_0 = 7.0752e-04
Validation auc = 0.723005
Epoch 210
Validation auc = 0.722066
Epoch 211
Validation auc = 0.730047
Epoch 212
Validation auc = 0.733333
Epoch 213
Validation auc = 0.732864
Epoch 214
Loss = 6.7272e-03, PNorm = 68.9794, GNorm = 0.1588, lr_0 = 7.0147e-04
Validation auc = 0.731455
Epoch 215
Validation auc = 0.732394
Epoch 216
Validation auc = 0.726761
Epoch 217
Validation auc = 0.724883
Epoch 218
Validation auc = 0.732394
Epoch 219
Loss = 5.1713e-03, PNorm = 69.0763, GNorm = 0.0926, lr_0 = 6.9548e-04
Validation auc = 0.728638
Epoch 220
Validation auc = 0.717371
Epoch 221
Validation auc = 0.711737
Epoch 222
Validation auc = 0.710329
Epoch 223
Validation auc = 0.707512
Epoch 224
Loss = 3.7796e-03, PNorm = 69.1315, GNorm = 0.1017, lr_0 = 6.8953e-04
Validation auc = 0.708920
Epoch 225
Validation auc = 0.707512
Epoch 226
Validation auc = 0.708451
Epoch 227
Validation auc = 0.714085
Epoch 228
Validation auc = 0.715023
Epoch 229
Loss = 5.3108e-03, PNorm = 69.1827, GNorm = 0.0750, lr_0 = 6.8364e-04
Validation auc = 0.715023
Epoch 230
Validation auc = 0.716901
Epoch 231
Validation auc = 0.719249
Epoch 232
Validation auc = 0.721596
Epoch 233
Validation auc = 0.718779
Epoch 234
Loss = 8.8043e-03, PNorm = 69.2515, GNorm = 0.5480, lr_0 = 6.7779e-04
Validation auc = 0.716432
Epoch 235
Validation auc = 0.712676
Epoch 236
Validation auc = 0.715962
Epoch 237
Validation auc = 0.716432
Epoch 238
Validation auc = 0.718310
Epoch 239
Loss = 1.0281e-02, PNorm = 69.3463, GNorm = 0.3157, lr_0 = 6.7200e-04
Validation auc = 0.716901
Epoch 240
Validation auc = 0.716901
Epoch 241
Validation auc = 0.728638
Epoch 242
Validation auc = 0.737089
Epoch 243
Validation auc = 0.738967
Epoch 244
Loss = 9.8665e-03, PNorm = 69.4611, GNorm = 0.2604, lr_0 = 6.6625e-04
Validation auc = 0.739437
Epoch 245
Validation auc = 0.743662
Epoch 246
Validation auc = 0.743662
Epoch 247
Validation auc = 0.741784
Epoch 248
Validation auc = 0.738967
Epoch 249
Loss = 2.0610e-02, PNorm = 69.6029, GNorm = 1.1442, lr_0 = 6.6056e-04
Validation auc = 0.738028
Epoch 250
Validation auc = 0.747418
Epoch 251
Validation auc = 0.746009
Epoch 252
Validation auc = 0.740376
Epoch 253
Validation auc = 0.736150
Epoch 254
Loss = 3.7384e-02, PNorm = 69.7296, GNorm = 0.9954, lr_0 = 6.5491e-04
Validation auc = 0.734272
Epoch 255
Validation auc = 0.732864
Epoch 256
Validation auc = 0.728169
Epoch 257
Validation auc = 0.724413
Epoch 258
Validation auc = 0.722535
Epoch 259
Loss = 2.3396e-02, PNorm = 69.8310, GNorm = 0.3153, lr_0 = 6.4931e-04
Validation auc = 0.717840
Epoch 260
Validation auc = 0.711268
Epoch 261
Validation auc = 0.711268
Epoch 262
Validation auc = 0.707042
Epoch 263
Validation auc = 0.706103
Epoch 264
Loss = 7.2895e-03, PNorm = 69.9195, GNorm = 0.0727, lr_0 = 6.4376e-04
Validation auc = 0.707512
Epoch 265
Validation auc = 0.709390
Epoch 266
Validation auc = 0.709859
Epoch 267
Validation auc = 0.710798
Epoch 268
Validation auc = 0.712676
Epoch 269
Loss = 1.0194e-02, PNorm = 69.9745, GNorm = 0.4589, lr_0 = 6.3826e-04
Validation auc = 0.710329
Epoch 270
Validation auc = 0.707512
Epoch 271
Validation auc = 0.706573
Epoch 272
Validation auc = 0.705164
Epoch 273
Validation auc = 0.705634
Epoch 274
Loss = 6.6826e-03, PNorm = 70.0311, GNorm = 0.0246, lr_0 = 6.3280e-04
Validation auc = 0.705164
Epoch 275
Validation auc = 0.707981
Epoch 276
Validation auc = 0.710329
Epoch 277
Validation auc = 0.711268
Epoch 278
Validation auc = 0.711737
Epoch 279
Loss = 7.9697e-03, PNorm = 70.1135, GNorm = 0.0502, lr_0 = 6.2739e-04
Validation auc = 0.717371
Epoch 280
Validation auc = 0.720657
Epoch 281
Validation auc = 0.724883
Epoch 282
Validation auc = 0.730516
Epoch 283
Validation auc = 0.735681
Epoch 284
Loss = 1.9665e-02, PNorm = 70.1886, GNorm = 0.0472, lr_0 = 6.2203e-04
Validation auc = 0.741784
Epoch 285
Validation auc = 0.747418
Epoch 286
Validation auc = 0.747887
Epoch 287
Validation auc = 0.746479
Epoch 288
Validation auc = 0.742254
Epoch 289
Loss = 6.8958e-03, PNorm = 70.2638, GNorm = 0.1723, lr_0 = 6.1671e-04
Validation auc = 0.737089
Epoch 290
Validation auc = 0.734272
Epoch 291
Validation auc = 0.733803
Epoch 292
Validation auc = 0.737089
Epoch 293
Validation auc = 0.739437
Epoch 294
Loss = 6.6029e-03, PNorm = 70.3327, GNorm = 0.2230, lr_0 = 6.1144e-04
Validation auc = 0.740845
Epoch 295
Validation auc = 0.743192
Epoch 296
Validation auc = 0.746009
Epoch 297
Validation auc = 0.745540
Epoch 298
Validation auc = 0.742254
Epoch 299
Loss = 5.7469e-03, PNorm = 70.3903, GNorm = 0.0382, lr_0 = 6.0621e-04
Validation auc = 0.737559
Model 0 best validation auc = 0.755869 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.898254
Ensemble test auc = 0.898254
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.709390
Epoch 1
Validation auc = 0.738967
Epoch 2
Validation auc = 0.731925
Epoch 3
Validation auc = 0.739437
Epoch 4
Loss = 4.9518e-01, PNorm = 63.7752, GNorm = 2.9378, lr_0 = 6.5000e-04
Validation auc = 0.722066
Epoch 5
Validation auc = 0.719249
Epoch 6
Validation auc = 0.715962
Epoch 7
Validation auc = 0.717371
Epoch 8
Validation auc = 0.717371
Epoch 9
Loss = 4.2799e-01, PNorm = 63.9306, GNorm = 3.6537, lr_0 = 9.9743e-04
Validation auc = 0.700939
Epoch 10
Validation auc = 0.692488
Epoch 11
Validation auc = 0.700939
Epoch 12
Validation auc = 0.701408
Epoch 13
Validation auc = 0.713146
Epoch 14
Loss = 3.4644e-01, PNorm = 64.1002, GNorm = 1.7907, lr_0 = 9.8890e-04
Validation auc = 0.723944
Epoch 15
Validation auc = 0.729108
Epoch 16
Validation auc = 0.731455
Epoch 17
Validation auc = 0.731925
Epoch 18
Validation auc = 0.738028
Epoch 19
Loss = 3.1962e-01, PNorm = 64.2294, GNorm = 2.3467, lr_0 = 9.8045e-04
Validation auc = 0.730986
Epoch 20
Validation auc = 0.711268
Epoch 21
Validation auc = 0.710329
Epoch 22
Validation auc = 0.710798
Epoch 23
Validation auc = 0.719718
Epoch 24
Loss = 2.0258e-01, PNorm = 64.3332, GNorm = 1.0552, lr_0 = 9.7207e-04
Validation auc = 0.714085
Epoch 25
Validation auc = 0.715962
Epoch 26
Validation auc = 0.722535
Epoch 27
Validation auc = 0.727230
Epoch 28
Validation auc = 0.727230
Epoch 29
Loss = 2.2447e-01, PNorm = 64.4256, GNorm = 1.4086, lr_0 = 9.6376e-04
Validation auc = 0.725352
Epoch 30
Validation auc = 0.724883
Epoch 31
Validation auc = 0.718310
Epoch 32
Validation auc = 0.712676
Epoch 33
Validation auc = 0.708451
Epoch 34
Loss = 1.4358e-01, PNorm = 64.5287, GNorm = 0.9654, lr_0 = 9.5552e-04
Validation auc = 0.711268
Epoch 35
Validation auc = 0.705634
Epoch 36
Validation auc = 0.716432
Epoch 37
Validation auc = 0.719718
Epoch 38
Validation auc = 0.712676
Epoch 39
Loss = 1.7182e-01, PNorm = 64.6230, GNorm = 1.4807, lr_0 = 9.4735e-04
Validation auc = 0.707981
Epoch 40
Validation auc = 0.705634
Epoch 41
Validation auc = 0.716432
Epoch 42
Validation auc = 0.713146
Epoch 43
Validation auc = 0.714554
Epoch 44
Loss = 1.6334e-01, PNorm = 64.7204, GNorm = 1.7445, lr_0 = 9.3925e-04
Validation auc = 0.727700
Epoch 45
Validation auc = 0.725352
Epoch 46
Validation auc = 0.730516
Epoch 47
Validation auc = 0.735681
Epoch 48
Validation auc = 0.734272
Epoch 49
Loss = 1.6446e-01, PNorm = 64.8320, GNorm = 0.8930, lr_0 = 9.3122e-04
Validation auc = 0.726761
Epoch 50
Validation auc = 0.726761
Epoch 51
Validation auc = 0.726761
Epoch 52
Validation auc = 0.723944
Epoch 53
Validation auc = 0.724883
Epoch 54
Loss = 2.2882e-01, PNorm = 64.9648, GNorm = 1.8744, lr_0 = 9.2326e-04
Validation auc = 0.721596
Epoch 55
Validation auc = 0.723944
Epoch 56
Validation auc = 0.722535
Epoch 57
Validation auc = 0.723005
Epoch 58
Validation auc = 0.717371
Epoch 59
Loss = 1.2082e-01, PNorm = 65.1116, GNorm = 1.0551, lr_0 = 9.1537e-04
Validation auc = 0.710329
Epoch 60
Validation auc = 0.700939
Epoch 61
Validation auc = 0.696714
Epoch 62
Validation auc = 0.701408
Epoch 63
Validation auc = 0.704225
Epoch 64
Loss = 1.8737e-01, PNorm = 65.2699, GNorm = 1.9807, lr_0 = 9.0754e-04
Validation auc = 0.702817
Epoch 65
Validation auc = 0.701408
Epoch 66
Validation auc = 0.700000
Epoch 67
Validation auc = 0.711737
Epoch 68
Validation auc = 0.729108
Epoch 69
Loss = 2.7385e-01, PNorm = 65.4067, GNorm = 2.8536, lr_0 = 8.9978e-04
Validation auc = 0.731455
Epoch 70
Validation auc = 0.727230
Epoch 71
Validation auc = 0.731455
Epoch 72
Validation auc = 0.736150
Epoch 73
Validation auc = 0.740845
Epoch 74
Loss = 9.4104e-02, PNorm = 65.5635, GNorm = 0.5343, lr_0 = 8.9209e-04
Validation auc = 0.741784
Epoch 75
Validation auc = 0.743192
Epoch 76
Validation auc = 0.731455
Epoch 77
Validation auc = 0.723944
Epoch 78
Validation auc = 0.720657
Epoch 79
Loss = 8.8443e-02, PNorm = 65.7243, GNorm = 0.6135, lr_0 = 8.8447e-04
Validation auc = 0.717371
Epoch 80
Validation auc = 0.716432
Epoch 81
Validation auc = 0.715962
Epoch 82
Validation auc = 0.709390
Epoch 83
Validation auc = 0.710329
Epoch 84
Loss = 1.0515e-01, PNorm = 65.8562, GNorm = 1.0135, lr_0 = 8.7691e-04
Validation auc = 0.714554
Epoch 85
Validation auc = 0.725352
Epoch 86
Validation auc = 0.725822
Epoch 87
Validation auc = 0.723944
Epoch 88
Validation auc = 0.725822
Epoch 89
Loss = 9.2569e-02, PNorm = 65.9909, GNorm = 1.1676, lr_0 = 8.6941e-04
Validation auc = 0.735681
Epoch 90
Validation auc = 0.738498
Epoch 91
Validation auc = 0.739437
Epoch 92
Validation auc = 0.741315
Epoch 93
Validation auc = 0.742254
Epoch 94
Loss = 7.0717e-02, PNorm = 66.0951, GNorm = 0.5323, lr_0 = 8.6198e-04
Validation auc = 0.741784
Epoch 95
Validation auc = 0.738967
Epoch 96
Validation auc = 0.741315
Epoch 97
Validation auc = 0.738967
Epoch 98
Validation auc = 0.739906
Epoch 99
Loss = 3.9417e-02, PNorm = 66.2373, GNorm = 0.3273, lr_0 = 8.5461e-04
Validation auc = 0.734272
Epoch 100
Validation auc = 0.729577
Epoch 101
Validation auc = 0.726291
Epoch 102
Validation auc = 0.719249
Epoch 103
Validation auc = 0.716901
Epoch 104
Loss = 4.6382e-02, PNorm = 66.3524, GNorm = 1.3538, lr_0 = 8.4730e-04
Validation auc = 0.708920
Epoch 105
Validation auc = 0.700939
Epoch 106
Validation auc = 0.701878
Epoch 107
Validation auc = 0.706573
Epoch 108
Validation auc = 0.715493
Epoch 109
Loss = 8.7523e-02, PNorm = 66.4749, GNorm = 1.3607, lr_0 = 8.4006e-04
Validation auc = 0.721596
Epoch 110
Validation auc = 0.718779
Epoch 111
Validation auc = 0.720657
Epoch 112
Validation auc = 0.728169
Epoch 113
Validation auc = 0.733333
Epoch 114
Loss = 5.1817e-02, PNorm = 66.6410, GNorm = 0.6847, lr_0 = 8.3288e-04
Validation auc = 0.735211
Epoch 115
Validation auc = 0.729577
Epoch 116
Validation auc = 0.720188
Epoch 117
Validation auc = 0.716432
Epoch 118
Validation auc = 0.714554
Epoch 119
Loss = 1.0095e-01, PNorm = 66.7808, GNorm = 1.5077, lr_0 = 8.2576e-04
Validation auc = 0.715493
Epoch 120
Validation auc = 0.715023
Epoch 121
Validation auc = 0.709390
Epoch 122
Validation auc = 0.712676
Epoch 123
Validation auc = 0.709859
Epoch 124
Loss = 5.1716e-02, PNorm = 66.9051, GNorm = 0.9031, lr_0 = 8.1870e-04
Validation auc = 0.707512
Epoch 125
Validation auc = 0.707042
Epoch 126
Validation auc = 0.708451
Epoch 127
Validation auc = 0.716901
Epoch 128
Validation auc = 0.716432
Epoch 129
Loss = 6.7302e-02, PNorm = 67.0347, GNorm = 0.6997, lr_0 = 8.1170e-04
Validation auc = 0.713146
Epoch 130
Validation auc = 0.708920
Epoch 131
Validation auc = 0.709390
Epoch 132
Validation auc = 0.721127
Epoch 133
Validation auc = 0.727700
Epoch 134
Loss = 4.8594e-02, PNorm = 67.1858, GNorm = 1.0109, lr_0 = 8.0476e-04
Validation auc = 0.730516
Epoch 135
Validation auc = 0.720657
Epoch 136
Validation auc = 0.700939
Epoch 137
Validation auc = 0.694836
Epoch 138
Validation auc = 0.692958
Epoch 139
Loss = 2.6308e-02, PNorm = 67.3097, GNorm = 0.1107, lr_0 = 7.9788e-04
Validation auc = 0.691549
Epoch 140
Validation auc = 0.692488
Epoch 141
Validation auc = 0.692019
Epoch 142
Validation auc = 0.696244
Epoch 143
Validation auc = 0.700000
Epoch 144
Loss = 3.9681e-02, PNorm = 67.4444, GNorm = 0.6223, lr_0 = 7.9106e-04
Validation auc = 0.700000
Epoch 145
Validation auc = 0.698122
Epoch 146
Validation auc = 0.699061
Epoch 147
Validation auc = 0.700469
Epoch 148
Validation auc = 0.724413
Epoch 149
Loss = 1.8023e-02, PNorm = 67.5781, GNorm = 0.6053, lr_0 = 7.8430e-04
Validation auc = 0.746479
Epoch 150
Validation auc = 0.723474
Epoch 151
Validation auc = 0.693897
Epoch 152
Validation auc = 0.696244
Epoch 153
Validation auc = 0.703286
Epoch 154
Loss = 3.7715e-02, PNorm = 67.7207, GNorm = 0.8438, lr_0 = 7.7759e-04
Validation auc = 0.707512
Epoch 155
Validation auc = 0.707981
Epoch 156
Validation auc = 0.706573
Epoch 157
Validation auc = 0.704225
Epoch 158
Validation auc = 0.703286
Epoch 159
Loss = 2.8584e-02, PNorm = 67.8662, GNorm = 0.1043, lr_0 = 7.7095e-04
Validation auc = 0.700469
Epoch 160
Validation auc = 0.695775
Epoch 161
Validation auc = 0.692019
Epoch 162
Validation auc = 0.693427
Epoch 163
Validation auc = 0.696714
Epoch 164
Loss = 2.7673e-02, PNorm = 68.0050, GNorm = 0.3163, lr_0 = 7.6436e-04
Validation auc = 0.696714
Epoch 165
Validation auc = 0.697183
Epoch 166
Validation auc = 0.691549
Epoch 167
Validation auc = 0.691549
Epoch 168
Validation auc = 0.702817
Epoch 169
Loss = 2.0282e-02, PNorm = 68.1555, GNorm = 0.4995, lr_0 = 7.5782e-04
Validation auc = 0.710329
Epoch 170
Validation auc = 0.711268
Epoch 171
Validation auc = 0.708451
Epoch 172
Validation auc = 0.704225
Epoch 173
Validation auc = 0.704695
Epoch 174
Loss = 3.7272e-02, PNorm = 68.2585, GNorm = 0.8721, lr_0 = 7.5134e-04
Validation auc = 0.700939
Epoch 175
Validation auc = 0.699061
Epoch 176
Validation auc = 0.707512
Epoch 177
Validation auc = 0.712207
Epoch 178
Validation auc = 0.711737
Epoch 179
Loss = 2.7678e-02, PNorm = 68.3669, GNorm = 0.4441, lr_0 = 7.4492e-04
Validation auc = 0.708451
Epoch 180
Validation auc = 0.705164
Epoch 181
Validation auc = 0.703286
Epoch 182
Validation auc = 0.702347
Epoch 183
Validation auc = 0.707042
Epoch 184
Loss = 1.3216e-02, PNorm = 68.4636, GNorm = 0.1408, lr_0 = 7.3855e-04
Validation auc = 0.707981
Epoch 185
Validation auc = 0.705634
Epoch 186
Validation auc = 0.702347
Epoch 187
Validation auc = 0.699061
Epoch 188
Validation auc = 0.696714
Epoch 189
Loss = 4.8417e-02, PNorm = 68.5420, GNorm = 0.8873, lr_0 = 7.3224e-04
Validation auc = 0.696714
Epoch 190
Validation auc = 0.695305
Epoch 191
Validation auc = 0.691549
Epoch 192
Validation auc = 0.692958
Epoch 193
Validation auc = 0.694366
Epoch 194
Loss = 1.5456e-02, PNorm = 68.6231, GNorm = 0.6987, lr_0 = 7.2598e-04
Validation auc = 0.693427
Epoch 195
Validation auc = 0.693897
Epoch 196
Validation auc = 0.696714
Epoch 197
Validation auc = 0.700939
Epoch 198
Validation auc = 0.705164
Epoch 199
Loss = 7.7102e-03, PNorm = 68.6973, GNorm = 0.1082, lr_0 = 7.1977e-04
Validation auc = 0.704225
Epoch 200
Validation auc = 0.704225
Epoch 201
Validation auc = 0.704225
Epoch 202
Validation auc = 0.694836
Epoch 203
Validation auc = 0.683568
Epoch 204
Loss = 1.6009e-02, PNorm = 68.7611, GNorm = 0.7583, lr_0 = 7.1362e-04
Validation auc = 0.678404
Epoch 205
Validation auc = 0.677934
Epoch 206
Validation auc = 0.681221
Epoch 207
Validation auc = 0.683099
Epoch 208
Validation auc = 0.683099
Epoch 209
Loss = 2.7693e-02, PNorm = 68.8300, GNorm = 0.1736, lr_0 = 7.0752e-04
Validation auc = 0.687324
Epoch 210
Validation auc = 0.689202
Epoch 211
Validation auc = 0.694366
Epoch 212
Validation auc = 0.700939
Epoch 213
Validation auc = 0.701878
Epoch 214
Loss = 3.4754e-03, PNorm = 68.8889, GNorm = 0.0447, lr_0 = 7.0147e-04
Validation auc = 0.703286
Epoch 215
Validation auc = 0.704225
Epoch 216
Validation auc = 0.704225
Epoch 217
Validation auc = 0.706103
Epoch 218
Validation auc = 0.706573
Epoch 219
Loss = 1.5187e-02, PNorm = 68.9564, GNorm = 0.2134, lr_0 = 6.9548e-04
Validation auc = 0.707512
Epoch 220
Validation auc = 0.706573
Epoch 221
Validation auc = 0.705164
Epoch 222
Validation auc = 0.706573
Epoch 223
Validation auc = 0.707042
Epoch 224
Loss = 8.2675e-03, PNorm = 69.0366, GNorm = 0.0864, lr_0 = 6.8953e-04
Validation auc = 0.713615
Epoch 225
Validation auc = 0.714554
Epoch 226
Validation auc = 0.709390
Epoch 227
Validation auc = 0.707042
Epoch 228
Validation auc = 0.702817
Epoch 229
Loss = 1.6086e-02, PNorm = 69.1047, GNorm = 0.8221, lr_0 = 6.8364e-04
Validation auc = 0.699061
Epoch 230
Validation auc = 0.699061
Epoch 231
Validation auc = 0.703286
Epoch 232
Validation auc = 0.705634
Epoch 233
Validation auc = 0.700000
Epoch 234
Loss = 1.8860e-02, PNorm = 69.1764, GNorm = 1.2589, lr_0 = 6.7779e-04
Validation auc = 0.697183
Epoch 235
Validation auc = 0.699531
Epoch 236
Validation auc = 0.700469
Epoch 237
Validation auc = 0.699061
Epoch 238
Validation auc = 0.695775
Epoch 239
Loss = 8.0994e-03, PNorm = 69.2940, GNorm = 0.1796, lr_0 = 6.7200e-04
Validation auc = 0.695305
Epoch 240
Validation auc = 0.693427
Epoch 241
Validation auc = 0.692958
Epoch 242
Validation auc = 0.689202
Epoch 243
Validation auc = 0.687793
Epoch 244
Loss = 7.6458e-03, PNorm = 69.3775, GNorm = 0.0815, lr_0 = 6.6625e-04
Validation auc = 0.685446
Epoch 245
Validation auc = 0.681221
Epoch 246
Validation auc = 0.675587
Epoch 247
Validation auc = 0.676056
Epoch 248
Validation auc = 0.675117
Epoch 249
Loss = 1.0991e-02, PNorm = 69.4453, GNorm = 0.0806, lr_0 = 6.6056e-04
Validation auc = 0.675587
Epoch 250
Validation auc = 0.676526
Epoch 251
Validation auc = 0.674178
Epoch 252
Validation auc = 0.672770
Epoch 253
Validation auc = 0.672300
Epoch 254
Loss = 3.6568e-03, PNorm = 69.4962, GNorm = 0.1609, lr_0 = 6.5491e-04
Validation auc = 0.671831
Epoch 255
Validation auc = 0.675117
Epoch 256
Validation auc = 0.678404
Epoch 257
Validation auc = 0.681690
Epoch 258
Validation auc = 0.684038
Epoch 259
Loss = 5.1220e-03, PNorm = 69.5287, GNorm = 0.0852, lr_0 = 6.4931e-04
Validation auc = 0.684507
Epoch 260
Validation auc = 0.686385
Epoch 261
Validation auc = 0.688263
Epoch 262
Validation auc = 0.696244
Epoch 263
Validation auc = 0.704225
Epoch 264
Loss = 4.7667e-03, PNorm = 69.5996, GNorm = 0.1241, lr_0 = 6.4376e-04
Validation auc = 0.706103
Epoch 265
Validation auc = 0.707512
Epoch 266
Validation auc = 0.707981
Epoch 267
Validation auc = 0.703756
Epoch 268
Validation auc = 0.698592
Epoch 269
Loss = 7.7296e-03, PNorm = 69.6567, GNorm = 0.1602, lr_0 = 6.3826e-04
Validation auc = 0.699531
Epoch 270
Validation auc = 0.699061
Epoch 271
Validation auc = 0.702817
Epoch 272
Validation auc = 0.707512
Epoch 273
Validation auc = 0.714554
Epoch 274
Loss = 2.7019e-02, PNorm = 69.7355, GNorm = 0.8544, lr_0 = 6.3280e-04
Validation auc = 0.716432
Epoch 275
Validation auc = 0.715493
Epoch 276
Validation auc = 0.716432
Epoch 277
Validation auc = 0.704695
Epoch 278
Validation auc = 0.696714
Epoch 279
Loss = 1.0067e-02, PNorm = 69.8343, GNorm = 0.5489, lr_0 = 6.2739e-04
Validation auc = 0.692958
Epoch 280
Validation auc = 0.693897
Epoch 281
Validation auc = 0.701878
Epoch 282
Validation auc = 0.700939
Epoch 283
Validation auc = 0.698122
Epoch 284
Loss = 9.2135e-03, PNorm = 69.9158, GNorm = 0.0707, lr_0 = 6.2203e-04
Validation auc = 0.699531
Epoch 285
Validation auc = 0.698122
Epoch 286
Validation auc = 0.695305
Epoch 287
Validation auc = 0.694836
Epoch 288
Validation auc = 0.693897
Epoch 289
Loss = 9.3315e-03, PNorm = 69.9707, GNorm = 0.1062, lr_0 = 6.1671e-04
Validation auc = 0.692958
Epoch 290
Validation auc = 0.692019
Epoch 291
Validation auc = 0.694836
Epoch 292
Validation auc = 0.696244
Epoch 293
Validation auc = 0.694836
Epoch 294
Loss = 8.1260e-03, PNorm = 70.0196, GNorm = 0.2073, lr_0 = 6.1144e-04
Validation auc = 0.696244
Epoch 295
Validation auc = 0.696244
Epoch 296
Validation auc = 0.697183
Epoch 297
Validation auc = 0.697183
Epoch 298
Validation auc = 0.695775
Epoch 299
Loss = 2.0506e-02, PNorm = 70.0782, GNorm = 0.6717, lr_0 = 6.0621e-04
Validation auc = 0.693897
Model 0 best validation auc = 0.746479 on epoch 149
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.921734
Ensemble test auc = 0.921734
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.680751
Epoch 1
Validation auc = 0.714085
Epoch 2
Validation auc = 0.682160
Epoch 3
Validation auc = 0.744131
Epoch 4
Loss = 6.1152e-01, PNorm = 63.7753, GNorm = 5.4803, lr_0 = 6.5000e-04
Validation auc = 0.734272
Epoch 5
Validation auc = 0.725352
Epoch 6
Validation auc = 0.748826
Epoch 7
Validation auc = 0.750235
Epoch 8
Validation auc = 0.740845
Epoch 9
Loss = 3.9916e-01, PNorm = 63.9373, GNorm = 1.9505, lr_0 = 9.9743e-04
Validation auc = 0.730986
Epoch 10
Validation auc = 0.731455
Epoch 11
Validation auc = 0.728169
Epoch 12
Validation auc = 0.720188
Epoch 13
Validation auc = 0.720188
Epoch 14
Loss = 2.6012e-01, PNorm = 64.1157, GNorm = 1.7372, lr_0 = 9.8890e-04
Validation auc = 0.721127
Epoch 15
Validation auc = 0.727230
Epoch 16
Validation auc = 0.737089
Epoch 17
Validation auc = 0.750704
Epoch 18
Validation auc = 0.751174
Epoch 19
Loss = 2.9273e-01, PNorm = 64.2424, GNorm = 1.3842, lr_0 = 9.8045e-04
Validation auc = 0.731925
Epoch 20
Validation auc = 0.732864
Epoch 21
Validation auc = 0.734272
Epoch 22
Validation auc = 0.731925
Epoch 23
Validation auc = 0.717371
Epoch 24
Loss = 2.0135e-01, PNorm = 64.3405, GNorm = 2.2218, lr_0 = 9.7207e-04
Validation auc = 0.714554
Epoch 25
Validation auc = 0.727700
Epoch 26
Validation auc = 0.738498
Epoch 27
Validation auc = 0.737089
Epoch 28
Validation auc = 0.740376
Epoch 29
Loss = 3.4562e-01, PNorm = 64.4315, GNorm = 1.3565, lr_0 = 9.6376e-04
Validation auc = 0.746948
Epoch 30
Validation auc = 0.752113
Epoch 31
Validation auc = 0.748826
Epoch 32
Validation auc = 0.740376
Epoch 33
Validation auc = 0.720657
Epoch 34
Loss = 1.7867e-01, PNorm = 64.5191, GNorm = 0.8839, lr_0 = 9.5552e-04
Validation auc = 0.708920
Epoch 35
Validation auc = 0.708451
Epoch 36
Validation auc = 0.719718
Epoch 37
Validation auc = 0.738028
Epoch 38
Validation auc = 0.738028
Epoch 39
Loss = 2.1457e-01, PNorm = 64.6187, GNorm = 1.3994, lr_0 = 9.4735e-04
Validation auc = 0.730986
Epoch 40
Validation auc = 0.725822
Epoch 41
Validation auc = 0.726291
Epoch 42
Validation auc = 0.726761
Epoch 43
Validation auc = 0.720657
Epoch 44
Loss = 1.8329e-01, PNorm = 64.7119, GNorm = 0.8518, lr_0 = 9.3925e-04
Validation auc = 0.715023
Epoch 45
Validation auc = 0.711737
Epoch 46
Validation auc = 0.712676
Epoch 47
Validation auc = 0.717840
Epoch 48
Validation auc = 0.715493
Epoch 49
Loss = 1.2032e-01, PNorm = 64.8114, GNorm = 1.1566, lr_0 = 9.3122e-04
Validation auc = 0.712676
Epoch 50
Validation auc = 0.714554
Epoch 51
Validation auc = 0.713615
Epoch 52
Validation auc = 0.711268
Epoch 53
Validation auc = 0.714554
Epoch 54
Loss = 1.2270e-01, PNorm = 64.9119, GNorm = 0.7997, lr_0 = 9.2326e-04
Validation auc = 0.722535
Epoch 55
Validation auc = 0.726761
Epoch 56
Validation auc = 0.731455
Epoch 57
Validation auc = 0.730047
Epoch 58
Validation auc = 0.728169
Epoch 59
Loss = 1.6560e-01, PNorm = 65.0207, GNorm = 1.2474, lr_0 = 9.1537e-04
Validation auc = 0.722535
Epoch 60
Validation auc = 0.708920
Epoch 61
Validation auc = 0.696244
Epoch 62
Validation auc = 0.694366
Epoch 63
Validation auc = 0.703756
Epoch 64
Loss = 9.3623e-02, PNorm = 65.1479, GNorm = 0.6709, lr_0 = 9.0754e-04
Validation auc = 0.710329
Epoch 65
Validation auc = 0.709859
Epoch 66
Validation auc = 0.713146
Epoch 67
Validation auc = 0.717371
Epoch 68
Validation auc = 0.715962
Epoch 69
Loss = 6.3222e-02, PNorm = 65.2904, GNorm = 0.7202, lr_0 = 8.9978e-04
Validation auc = 0.719718
Epoch 70
Validation auc = 0.724883
Epoch 71
Validation auc = 0.723944
Epoch 72
Validation auc = 0.715023
Epoch 73
Validation auc = 0.707512
Epoch 74
Loss = 5.3108e-02, PNorm = 65.4408, GNorm = 0.3811, lr_0 = 8.9209e-04
Validation auc = 0.704225
Epoch 75
Validation auc = 0.709859
Epoch 76
Validation auc = 0.715023
Epoch 77
Validation auc = 0.713146
Epoch 78
Validation auc = 0.709390
Epoch 79
Loss = 2.0685e-01, PNorm = 65.5710, GNorm = 1.8845, lr_0 = 8.8447e-04
Validation auc = 0.714085
Epoch 80
Validation auc = 0.718310
Epoch 81
Validation auc = 0.719249
Epoch 82
Validation auc = 0.715962
Epoch 83
Validation auc = 0.712676
Epoch 84
Loss = 1.4782e-01, PNorm = 65.7097, GNorm = 1.2768, lr_0 = 8.7691e-04
Validation auc = 0.717371
Epoch 85
Validation auc = 0.715962
Epoch 86
Validation auc = 0.715023
Epoch 87
Validation auc = 0.710329
Epoch 88
Validation auc = 0.707981
Epoch 89
Loss = 1.0491e-01, PNorm = 65.8431, GNorm = 1.2177, lr_0 = 8.6941e-04
Validation auc = 0.713615
Epoch 90
Validation auc = 0.720657
Epoch 91
Validation auc = 0.723944
Epoch 92
Validation auc = 0.723005
Epoch 93
Validation auc = 0.723005
Epoch 94
Loss = 5.1651e-02, PNorm = 65.9841, GNorm = 0.4638, lr_0 = 8.6198e-04
Validation auc = 0.721127
Epoch 95
Validation auc = 0.718310
Epoch 96
Validation auc = 0.714554
Epoch 97
Validation auc = 0.713615
Epoch 98
Validation auc = 0.714085
Epoch 99
Loss = 5.4440e-02, PNorm = 66.1142, GNorm = 0.8691, lr_0 = 8.5461e-04
Validation auc = 0.707512
Epoch 100
Validation auc = 0.705634
Epoch 101
Validation auc = 0.700939
Epoch 102
Validation auc = 0.701878
Epoch 103
Validation auc = 0.694366
Epoch 104
Loss = 6.0315e-02, PNorm = 66.2208, GNorm = 0.4813, lr_0 = 8.4730e-04
Validation auc = 0.691080
Epoch 105
Validation auc = 0.686854
Epoch 106
Validation auc = 0.691080
Epoch 107
Validation auc = 0.696714
Epoch 108
Validation auc = 0.692958
Epoch 109
Loss = 4.3848e-02, PNorm = 66.3495, GNorm = 0.4984, lr_0 = 8.4006e-04
Validation auc = 0.690610
Epoch 110
Validation auc = 0.687793
Epoch 111
Validation auc = 0.690610
Epoch 112
Validation auc = 0.698122
Epoch 113
Validation auc = 0.700000
Epoch 114
Loss = 4.3095e-02, PNorm = 66.4845, GNorm = 0.7873, lr_0 = 8.3288e-04
Validation auc = 0.701878
Epoch 115
Validation auc = 0.699531
Epoch 116
Validation auc = 0.697653
Epoch 117
Validation auc = 0.708451
Epoch 118
Validation auc = 0.705164
Epoch 119
Loss = 6.8642e-02, PNorm = 66.6306, GNorm = 0.8613, lr_0 = 8.2576e-04
Validation auc = 0.701408
Epoch 120
Validation auc = 0.699531
Epoch 121
Validation auc = 0.694366
Epoch 122
Validation auc = 0.692958
Epoch 123
Validation auc = 0.690141
Epoch 124
Loss = 4.3005e-02, PNorm = 66.7795, GNorm = 0.8044, lr_0 = 8.1870e-04
Validation auc = 0.688732
Epoch 125
Validation auc = 0.684977
Epoch 126
Validation auc = 0.676995
Epoch 127
Validation auc = 0.671362
Epoch 128
Validation auc = 0.668075
Epoch 129
Loss = 4.3054e-02, PNorm = 66.9453, GNorm = 0.2036, lr_0 = 8.1170e-04
Validation auc = 0.679812
Epoch 130
Validation auc = 0.681690
Epoch 131
Validation auc = 0.676526
Epoch 132
Validation auc = 0.677934
Epoch 133
Validation auc = 0.682160
Epoch 134
Loss = 2.0493e-02, PNorm = 67.0863, GNorm = 0.5150, lr_0 = 8.0476e-04
Validation auc = 0.681221
Epoch 135
Validation auc = 0.689202
Epoch 136
Validation auc = 0.691549
Epoch 137
Validation auc = 0.688263
Epoch 138
Validation auc = 0.683099
Epoch 139
Loss = 3.1476e-02, PNorm = 67.2264, GNorm = 0.4808, lr_0 = 7.9788e-04
Validation auc = 0.693427
Epoch 140
Validation auc = 0.685915
Epoch 141
Validation auc = 0.684038
Epoch 142
Validation auc = 0.681221
Epoch 143
Validation auc = 0.679812
Epoch 144
Loss = 7.0293e-02, PNorm = 67.3819, GNorm = 0.7464, lr_0 = 7.9106e-04
Validation auc = 0.684507
Epoch 145
Validation auc = 0.684038
Epoch 146
Validation auc = 0.686385
Epoch 147
Validation auc = 0.688263
Epoch 148
Validation auc = 0.700939
Epoch 149
Loss = 4.8915e-02, PNorm = 67.5751, GNorm = 0.7775, lr_0 = 7.8430e-04
Validation auc = 0.713146
Epoch 150
Validation auc = 0.693897
Epoch 151
Validation auc = 0.690610
Epoch 152
Validation auc = 0.689202
Epoch 153
Validation auc = 0.691549
Epoch 154
Loss = 5.2497e-02, PNorm = 67.7276, GNorm = 0.4313, lr_0 = 7.7759e-04
Validation auc = 0.702347
Epoch 155
Validation auc = 0.710798
Epoch 156
Validation auc = 0.707042
Epoch 157
Validation auc = 0.702347
Epoch 158
Validation auc = 0.699061
Epoch 159
Loss = 1.9189e-02, PNorm = 67.8902, GNorm = 0.4433, lr_0 = 7.7095e-04
Validation auc = 0.694366
Epoch 160
Validation auc = 0.698122
Epoch 161
Validation auc = 0.707042
Epoch 162
Validation auc = 0.710329
Epoch 163
Validation auc = 0.707981
Epoch 164
Loss = 9.7376e-03, PNorm = 68.0137, GNorm = 0.1884, lr_0 = 7.6436e-04
Validation auc = 0.701878
Epoch 165
Validation auc = 0.701408
Epoch 166
Validation auc = 0.702347
Epoch 167
Validation auc = 0.707512
Epoch 168
Validation auc = 0.710798
Epoch 169
Loss = 8.5391e-03, PNorm = 68.1775, GNorm = 0.0623, lr_0 = 7.5782e-04
Validation auc = 0.713615
Epoch 170
Validation auc = 0.711737
Epoch 171
Validation auc = 0.711737
Epoch 172
Validation auc = 0.708920
Epoch 173
Validation auc = 0.711268
Epoch 174
Loss = 3.6909e-02, PNorm = 68.2948, GNorm = 0.3113, lr_0 = 7.5134e-04
Validation auc = 0.709390
Epoch 175
Validation auc = 0.703756
Epoch 176
Validation auc = 0.704695
Epoch 177
Validation auc = 0.707512
Epoch 178
Validation auc = 0.706573
Epoch 179
Loss = 1.6829e-02, PNorm = 68.4134, GNorm = 0.2667, lr_0 = 7.4492e-04
Validation auc = 0.701878
Epoch 180
Validation auc = 0.697183
Epoch 181
Validation auc = 0.688263
Epoch 182
Validation auc = 0.683568
Epoch 183
Validation auc = 0.684038
Epoch 184
Loss = 1.8180e-02, PNorm = 68.5126, GNorm = 0.3637, lr_0 = 7.3855e-04
Validation auc = 0.695775
Epoch 185
Validation auc = 0.695305
Epoch 186
Validation auc = 0.682629
Epoch 187
Validation auc = 0.687324
Epoch 188
Validation auc = 0.688732
Epoch 189
Loss = 1.4027e-02, PNorm = 68.5758, GNorm = 0.3164, lr_0 = 7.3224e-04
Validation auc = 0.683568
Epoch 190
Validation auc = 0.679812
Epoch 191
Validation auc = 0.686385
Epoch 192
Validation auc = 0.687324
Epoch 193
Validation auc = 0.684507
Epoch 194
Loss = 1.2984e-02, PNorm = 68.6667, GNorm = 0.4432, lr_0 = 7.2598e-04
Validation auc = 0.684507
Epoch 195
Validation auc = 0.684507
Epoch 196
Validation auc = 0.684507
Epoch 197
Validation auc = 0.684507
Epoch 198
Validation auc = 0.689202
Epoch 199
Loss = 6.9453e-03, PNorm = 68.7484, GNorm = 0.0320, lr_0 = 7.1977e-04
Validation auc = 0.697183
Epoch 200
Validation auc = 0.702817
Epoch 201
Validation auc = 0.708451
Epoch 202
Validation auc = 0.707512
Epoch 203
Validation auc = 0.715493
Epoch 204
Loss = 4.2701e-03, PNorm = 68.8386, GNorm = 0.1294, lr_0 = 7.1362e-04
Validation auc = 0.722535
Epoch 205
Validation auc = 0.729577
Epoch 206
Validation auc = 0.725352
Epoch 207
Validation auc = 0.725352
Epoch 208
Validation auc = 0.714554
Epoch 209
Loss = 1.1679e-02, PNorm = 68.9324, GNorm = 0.3086, lr_0 = 7.0752e-04
Validation auc = 0.707512
Epoch 210
Validation auc = 0.702347
Epoch 211
Validation auc = 0.696244
Epoch 212
Validation auc = 0.692958
Epoch 213
Validation auc = 0.690141
Epoch 214
Loss = 9.2547e-03, PNorm = 69.0208, GNorm = 0.3246, lr_0 = 7.0147e-04
Validation auc = 0.687324
Epoch 215
Validation auc = 0.687793
Epoch 216
Validation auc = 0.685446
Epoch 217
Validation auc = 0.682160
Epoch 218
Validation auc = 0.684977
Epoch 219
Loss = 4.6828e-03, PNorm = 69.0824, GNorm = 0.1992, lr_0 = 6.9548e-04
Validation auc = 0.685915
Epoch 220
Validation auc = 0.693897
Epoch 221
Validation auc = 0.697653
Epoch 222
Validation auc = 0.700000
Epoch 223
Validation auc = 0.702347
Epoch 224
Loss = 1.0790e-02, PNorm = 69.1433, GNorm = 0.1944, lr_0 = 6.8953e-04
Validation auc = 0.699531
Epoch 225
Validation auc = 0.694836
Epoch 226
Validation auc = 0.682160
Epoch 227
Validation auc = 0.684038
Epoch 228
Validation auc = 0.685446
Epoch 229
Loss = 4.0661e-03, PNorm = 69.1880, GNorm = 0.1300, lr_0 = 6.8364e-04
Validation auc = 0.687793
Epoch 230
Validation auc = 0.685446
Epoch 231
Validation auc = 0.678404
Epoch 232
Validation auc = 0.676056
Epoch 233
Validation auc = 0.673709
Epoch 234
Loss = 4.6574e-03, PNorm = 69.2512, GNorm = 0.1216, lr_0 = 6.7779e-04
Validation auc = 0.673239
Epoch 235
Validation auc = 0.671831
Epoch 236
Validation auc = 0.673239
Epoch 237
Validation auc = 0.674648
Epoch 238
Validation auc = 0.674648
Epoch 239
Loss = 9.5673e-03, PNorm = 69.3000, GNorm = 0.2868, lr_0 = 6.7200e-04
Validation auc = 0.676526
Epoch 240
Validation auc = 0.676995
Epoch 241
Validation auc = 0.677934
Epoch 242
Validation auc = 0.675587
Epoch 243
Validation auc = 0.675117
Epoch 244
Loss = 1.1924e-02, PNorm = 69.3573, GNorm = 0.5896, lr_0 = 6.6625e-04
Validation auc = 0.675117
Epoch 245
Validation auc = 0.687793
Epoch 246
Validation auc = 0.693427
Epoch 247
Validation auc = 0.698592
Epoch 248
Validation auc = 0.702347
Epoch 249
Loss = 5.3902e-03, PNorm = 69.4020, GNorm = 0.1402, lr_0 = 6.6056e-04
Validation auc = 0.700000
Epoch 250
Validation auc = 0.696714
Epoch 251
Validation auc = 0.695775
Epoch 252
Validation auc = 0.696244
Epoch 253
Validation auc = 0.694366
Epoch 254
Loss = 2.7671e-03, PNorm = 69.4575, GNorm = 0.0906, lr_0 = 6.5491e-04
Validation auc = 0.696244
Epoch 255
Validation auc = 0.695305
Epoch 256
Validation auc = 0.700469
Epoch 257
Validation auc = 0.700469
Epoch 258
Validation auc = 0.700939
Epoch 259
Loss = 1.7079e-03, PNorm = 69.5138, GNorm = 0.0610, lr_0 = 6.4931e-04
Validation auc = 0.700000
Epoch 260
Validation auc = 0.697653
Epoch 261
Validation auc = 0.695775
Epoch 262
Validation auc = 0.691549
Epoch 263
Validation auc = 0.687324
Epoch 264
Loss = 1.2916e-02, PNorm = 69.5645, GNorm = 0.5168, lr_0 = 6.4376e-04
Validation auc = 0.687793
Epoch 265
Validation auc = 0.686854
Epoch 266
Validation auc = 0.689671
Epoch 267
Validation auc = 0.692958
Epoch 268
Validation auc = 0.698122
Epoch 269
Loss = 5.7896e-03, PNorm = 69.6182, GNorm = 0.1783, lr_0 = 6.3826e-04
Validation auc = 0.700000
Epoch 270
Validation auc = 0.698122
Epoch 271
Validation auc = 0.693897
Epoch 272
Validation auc = 0.691080
Epoch 273
Validation auc = 0.693427
Epoch 274
Loss = 1.8291e-03, PNorm = 69.6686, GNorm = 0.0837, lr_0 = 6.3280e-04
Validation auc = 0.693427
Epoch 275
Validation auc = 0.691080
Epoch 276
Validation auc = 0.694836
Epoch 277
Validation auc = 0.695305
Epoch 278
Validation auc = 0.700000
Epoch 279
Loss = 7.1111e-03, PNorm = 69.7212, GNorm = 0.3141, lr_0 = 6.2739e-04
Validation auc = 0.700469
Epoch 280
Validation auc = 0.701878
Epoch 281
Validation auc = 0.698122
Epoch 282
Validation auc = 0.698122
Epoch 283
Validation auc = 0.694836
Epoch 284
Loss = 2.8478e-03, PNorm = 69.7703, GNorm = 0.0343, lr_0 = 6.2203e-04
Validation auc = 0.693897
Epoch 285
Validation auc = 0.693897
Epoch 286
Validation auc = 0.697183
Epoch 287
Validation auc = 0.700000
Epoch 288
Validation auc = 0.700939
Epoch 289
Loss = 3.5883e-03, PNorm = 69.8179, GNorm = 0.2063, lr_0 = 6.1671e-04
Validation auc = 0.700000
Epoch 290
Validation auc = 0.699061
Epoch 291
Validation auc = 0.697653
Epoch 292
Validation auc = 0.693427
Epoch 293
Validation auc = 0.690610
Epoch 294
Loss = 4.9883e-03, PNorm = 69.8518, GNorm = 0.1991, lr_0 = 6.1144e-04
Validation auc = 0.689671
Epoch 295
Validation auc = 0.688732
Epoch 296
Validation auc = 0.686385
Epoch 297
Validation auc = 0.688732
Epoch 298
Validation auc = 0.688732
Epoch 299
Loss = 1.7641e-03, PNorm = 69.8798, GNorm = 0.1206, lr_0 = 6.0621e-04
Validation auc = 0.689671
Model 0 best validation auc = 0.752113 on epoch 30
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.873570
Ensemble test auc = 0.873570
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.672300
Epoch 1
Validation auc = 0.716432
Epoch 2
Validation auc = 0.692019
Epoch 3
Validation auc = 0.707981
Epoch 4
Loss = 5.5398e-01, PNorm = 63.7795, GNorm = 2.0934, lr_0 = 6.5000e-04
Validation auc = 0.691080
Epoch 5
Validation auc = 0.723005
Epoch 6
Validation auc = 0.724413
Epoch 7
Validation auc = 0.739906
Epoch 8
Validation auc = 0.720657
Epoch 9
Loss = 4.3778e-01, PNorm = 63.9482, GNorm = 2.5696, lr_0 = 9.9743e-04
Validation auc = 0.708451
Epoch 10
Validation auc = 0.708920
Epoch 11
Validation auc = 0.707512
Epoch 12
Validation auc = 0.702817
Epoch 13
Validation auc = 0.711268
Epoch 14
Loss = 3.4996e-01, PNorm = 64.1403, GNorm = 1.5084, lr_0 = 9.8890e-04
Validation auc = 0.724883
Epoch 15
Validation auc = 0.719249
Epoch 16
Validation auc = 0.711268
Epoch 17
Validation auc = 0.711268
Epoch 18
Validation auc = 0.708451
Epoch 19
Loss = 2.2571e-01, PNorm = 64.2776, GNorm = 1.1709, lr_0 = 9.8045e-04
Validation auc = 0.707042
Epoch 20
Validation auc = 0.700939
Epoch 21
Validation auc = 0.707042
Epoch 22
Validation auc = 0.711268
Epoch 23
Validation auc = 0.710329
Epoch 24
Loss = 3.4964e-01, PNorm = 64.3846, GNorm = 2.2411, lr_0 = 9.7207e-04
Validation auc = 0.712676
Epoch 25
Validation auc = 0.715962
Epoch 26
Validation auc = 0.691080
Epoch 27
Validation auc = 0.681221
Epoch 28
Validation auc = 0.680751
Epoch 29
Loss = 1.8685e-01, PNorm = 64.4873, GNorm = 1.1687, lr_0 = 9.6376e-04
Validation auc = 0.690610
Epoch 30
Validation auc = 0.693897
Epoch 31
Validation auc = 0.690610
Epoch 32
Validation auc = 0.697183
Epoch 33
Validation auc = 0.702347
Epoch 34
Loss = 1.7908e-01, PNorm = 64.5908, GNorm = 1.7848, lr_0 = 9.5552e-04
Validation auc = 0.707981
Epoch 35
Validation auc = 0.711737
Epoch 36
Validation auc = 0.715023
Epoch 37
Validation auc = 0.713615
Epoch 38
Validation auc = 0.702347
Epoch 39
Loss = 2.2847e-01, PNorm = 64.6998, GNorm = 2.0294, lr_0 = 9.4735e-04
Validation auc = 0.715962
Epoch 40
Validation auc = 0.728638
Epoch 41
Validation auc = 0.731925
Epoch 42
Validation auc = 0.732394
Epoch 43
Validation auc = 0.727700
Epoch 44
Loss = 2.3866e-01, PNorm = 64.8083, GNorm = 1.0000, lr_0 = 9.3925e-04
Validation auc = 0.724883
Epoch 45
Validation auc = 0.721127
Epoch 46
Validation auc = 0.709859
Epoch 47
Validation auc = 0.698592
Epoch 48
Validation auc = 0.689202
Epoch 49
Loss = 1.6538e-01, PNorm = 64.9334, GNorm = 1.4352, lr_0 = 9.3122e-04
Validation auc = 0.690610
Epoch 50
Validation auc = 0.700000
Epoch 51
Validation auc = 0.702347
Epoch 52
Validation auc = 0.699531
Epoch 53
Validation auc = 0.693897
Epoch 54
Loss = 1.0095e-01, PNorm = 65.0735, GNorm = 1.0493, lr_0 = 9.2326e-04
Validation auc = 0.694366
Epoch 55
Validation auc = 0.694836
Epoch 56
Validation auc = 0.696244
Epoch 57
Validation auc = 0.697183
Epoch 58
Validation auc = 0.703286
Epoch 59
Loss = 1.0890e-01, PNorm = 65.2129, GNorm = 0.6435, lr_0 = 9.1537e-04
Validation auc = 0.705164
Epoch 60
Validation auc = 0.700000
Epoch 61
Validation auc = 0.694836
Epoch 62
Validation auc = 0.699531
Epoch 63
Validation auc = 0.693427
Epoch 64
Loss = 8.3896e-02, PNorm = 65.3503, GNorm = 0.6504, lr_0 = 9.0754e-04
Validation auc = 0.689202
Epoch 65
Validation auc = 0.690141
Epoch 66
Validation auc = 0.693897
Epoch 67
Validation auc = 0.694366
Epoch 68
Validation auc = 0.682160
Epoch 69
Loss = 1.4811e-01, PNorm = 65.4984, GNorm = 1.3291, lr_0 = 8.9978e-04
Validation auc = 0.708451
Epoch 70
Validation auc = 0.708451
Epoch 71
Validation auc = 0.703286
Epoch 72
Validation auc = 0.693897
Epoch 73
Validation auc = 0.694836
Epoch 74
Loss = 8.2065e-02, PNorm = 65.6438, GNorm = 1.3540, lr_0 = 8.9209e-04
Validation auc = 0.695775
Epoch 75
Validation auc = 0.699531
Epoch 76
Validation auc = 0.697183
Epoch 77
Validation auc = 0.693427
Epoch 78
Validation auc = 0.692019
Epoch 79
Loss = 1.2079e-01, PNorm = 65.7878, GNorm = 1.5049, lr_0 = 8.8447e-04
Validation auc = 0.693427
Epoch 80
Validation auc = 0.695775
Epoch 81
Validation auc = 0.692958
Epoch 82
Validation auc = 0.692019
Epoch 83
Validation auc = 0.687324
Epoch 84
Loss = 4.8740e-02, PNorm = 65.9418, GNorm = 0.7334, lr_0 = 8.7691e-04
Validation auc = 0.689202
Epoch 85
Validation auc = 0.693897
Epoch 86
Validation auc = 0.706573
Epoch 87
Validation auc = 0.703756
Epoch 88
Validation auc = 0.697653
Epoch 89
Loss = 1.4774e-01, PNorm = 66.0657, GNorm = 1.1878, lr_0 = 8.6941e-04
Validation auc = 0.696244
Epoch 90
Validation auc = 0.695775
Epoch 91
Validation auc = 0.698122
Epoch 92
Validation auc = 0.694836
Epoch 93
Validation auc = 0.691549
Epoch 94
Loss = 6.5076e-02, PNorm = 66.2140, GNorm = 1.2766, lr_0 = 8.6198e-04
Validation auc = 0.691080
Epoch 95
Validation auc = 0.691549
Epoch 96
Validation auc = 0.689202
Epoch 97
Validation auc = 0.691080
Epoch 98
Validation auc = 0.686854
Epoch 99
Loss = 5.3223e-02, PNorm = 66.3838, GNorm = 0.5325, lr_0 = 8.5461e-04
Validation auc = 0.683568
Epoch 100
Validation auc = 0.680282
Epoch 101
Validation auc = 0.680282
Epoch 102
Validation auc = 0.682629
Epoch 103
Validation auc = 0.684038
Epoch 104
Loss = 4.4369e-02, PNorm = 66.5362, GNorm = 0.7593, lr_0 = 8.4730e-04
Validation auc = 0.684038
Epoch 105
Validation auc = 0.693897
Epoch 106
Validation auc = 0.706103
Epoch 107
Validation auc = 0.707512
Epoch 108
Validation auc = 0.706573
Epoch 109
Loss = 4.0726e-02, PNorm = 66.6710, GNorm = 0.4644, lr_0 = 8.4006e-04
Validation auc = 0.701408
Epoch 110
Validation auc = 0.702817
Epoch 111
Validation auc = 0.714085
Epoch 112
Validation auc = 0.721127
Epoch 113
Validation auc = 0.712676
Epoch 114
Loss = 2.9091e-02, PNorm = 66.8069, GNorm = 0.4929, lr_0 = 8.3288e-04
Validation auc = 0.706103
Epoch 115
Validation auc = 0.701408
Epoch 116
Validation auc = 0.702817
Epoch 117
Validation auc = 0.703286
Epoch 118
Validation auc = 0.698122
Epoch 119
Loss = 3.8278e-02, PNorm = 66.9683, GNorm = 0.3544, lr_0 = 8.2576e-04
Validation auc = 0.689671
Epoch 120
Validation auc = 0.693427
Epoch 121
Validation auc = 0.705634
Epoch 122
Validation auc = 0.706103
Epoch 123
Validation auc = 0.704695
Epoch 124
Loss = 3.8769e-02, PNorm = 67.1025, GNorm = 0.4561, lr_0 = 8.1870e-04
Validation auc = 0.698122
Epoch 125
Validation auc = 0.698122
Epoch 126
Validation auc = 0.692019
Epoch 127
Validation auc = 0.690141
Epoch 128
Validation auc = 0.693427
Epoch 129
Loss = 4.0131e-02, PNorm = 67.2366, GNorm = 0.5087, lr_0 = 8.1170e-04
Validation auc = 0.697183
Epoch 130
Validation auc = 0.696244
Epoch 131
Validation auc = 0.694836
Epoch 132
Validation auc = 0.690141
Epoch 133
Validation auc = 0.691080
Epoch 134
Loss = 5.5468e-02, PNorm = 67.3909, GNorm = 0.3201, lr_0 = 8.0476e-04
Validation auc = 0.689202
Epoch 135
Validation auc = 0.687324
Epoch 136
Validation auc = 0.678404
Epoch 137
Validation auc = 0.663850
Epoch 138
Validation auc = 0.666197
Epoch 139
Loss = 4.1028e-02, PNorm = 67.5451, GNorm = 0.3382, lr_0 = 7.9788e-04
Validation auc = 0.680282
Epoch 140
Validation auc = 0.677465
Epoch 141
Validation auc = 0.665258
Epoch 142
Validation auc = 0.665728
Epoch 143
Validation auc = 0.669014
Epoch 144
Loss = 3.3310e-02, PNorm = 67.6945, GNorm = 0.7524, lr_0 = 7.9106e-04
Validation auc = 0.673709
Epoch 145
Validation auc = 0.681221
Epoch 146
Validation auc = 0.685446
Epoch 147
Validation auc = 0.682160
Epoch 148
Validation auc = 0.688263
Epoch 149
Loss = 4.7745e-02, PNorm = 67.8204, GNorm = 1.0113, lr_0 = 7.8430e-04
Validation auc = 0.692958
Epoch 150
Validation auc = 0.692958
Epoch 151
Validation auc = 0.692488
Epoch 152
Validation auc = 0.689202
Epoch 153
Validation auc = 0.691080
Epoch 154
Loss = 1.4409e-02, PNorm = 67.9222, GNorm = 0.2417, lr_0 = 7.7759e-04
Validation auc = 0.688732
Epoch 155
Validation auc = 0.686854
Epoch 156
Validation auc = 0.684507
Epoch 157
Validation auc = 0.681221
Epoch 158
Validation auc = 0.679812
Epoch 159
Loss = 1.1660e-02, PNorm = 68.0195, GNorm = 0.0747, lr_0 = 7.7095e-04
Validation auc = 0.678873
Epoch 160
Validation auc = 0.676526
Epoch 161
Validation auc = 0.676995
Epoch 162
Validation auc = 0.676526
Epoch 163
Validation auc = 0.678404
Epoch 164
Loss = 3.5174e-02, PNorm = 68.0963, GNorm = 0.1459, lr_0 = 7.6436e-04
Validation auc = 0.679343
Epoch 165
Validation auc = 0.678873
Epoch 166
Validation auc = 0.676995
Epoch 167
Validation auc = 0.677465
Epoch 168
Validation auc = 0.681690
Epoch 169
Loss = 8.7546e-03, PNorm = 68.1902, GNorm = 0.2518, lr_0 = 7.5782e-04
Validation auc = 0.679812
Epoch 170
Validation auc = 0.687324
Epoch 171
Validation auc = 0.695775
Epoch 172
Validation auc = 0.678404
Epoch 173
Validation auc = 0.661502
Epoch 174
Loss = 1.7563e-02, PNorm = 68.2809, GNorm = 0.5750, lr_0 = 7.5134e-04
Validation auc = 0.647887
Epoch 175
Validation auc = 0.669953
Epoch 176
Validation auc = 0.676995
Epoch 177
Validation auc = 0.683568
Epoch 178
Validation auc = 0.687793
Epoch 179
Loss = 8.7868e-03, PNorm = 68.4207, GNorm = 0.3015, lr_0 = 7.4492e-04
Validation auc = 0.689671
Epoch 180
Validation auc = 0.692019
Epoch 181
Validation auc = 0.695775
Epoch 182
Validation auc = 0.702817
Epoch 183
Validation auc = 0.710329
Epoch 184
Loss = 4.1425e-02, PNorm = 68.5848, GNorm = 0.2240, lr_0 = 7.3855e-04
Validation auc = 0.703286
Epoch 185
Validation auc = 0.701878
Epoch 186
Validation auc = 0.696244
Epoch 187
Validation auc = 0.692958
Epoch 188
Validation auc = 0.693897
Epoch 189
Loss = 1.2263e-02, PNorm = 68.7156, GNorm = 0.1470, lr_0 = 7.3224e-04
Validation auc = 0.691549
Epoch 190
Validation auc = 0.692019
Epoch 191
Validation auc = 0.688732
Epoch 192
Validation auc = 0.688263
Epoch 193
Validation auc = 0.688263
Epoch 194
Loss = 2.6844e-02, PNorm = 68.7993, GNorm = 0.6313, lr_0 = 7.2598e-04
Validation auc = 0.689202
Epoch 195
Validation auc = 0.689202
Epoch 196
Validation auc = 0.697183
Epoch 197
Validation auc = 0.706573
Epoch 198
Validation auc = 0.717371
Epoch 199
Loss = 1.3627e-02, PNorm = 68.8825, GNorm = 0.5747, lr_0 = 7.1977e-04
Validation auc = 0.725352
Epoch 200
Validation auc = 0.730516
Epoch 201
Validation auc = 0.731925
Epoch 202
Validation auc = 0.730047
Epoch 203
Validation auc = 0.731925
Epoch 204
Loss = 3.1566e-02, PNorm = 68.9772, GNorm = 0.1078, lr_0 = 7.1362e-04
Validation auc = 0.730986
Epoch 205
Validation auc = 0.733803
Epoch 206
Validation auc = 0.734742
Epoch 207
Validation auc = 0.736150
Epoch 208
Validation auc = 0.727700
Epoch 209
Loss = 2.1001e-02, PNorm = 69.0683, GNorm = 0.4540, lr_0 = 7.0752e-04
Validation auc = 0.716432
Epoch 210
Validation auc = 0.706103
Epoch 211
Validation auc = 0.703756
Epoch 212
Validation auc = 0.699531
Epoch 213
Validation auc = 0.695305
Epoch 214
Loss = 1.0327e-02, PNorm = 69.1323, GNorm = 0.5581, lr_0 = 7.0147e-04
Validation auc = 0.698122
Epoch 215
Validation auc = 0.705164
Epoch 216
Validation auc = 0.713146
Epoch 217
Validation auc = 0.715962
Epoch 218
Validation auc = 0.716901
Epoch 219
Loss = 4.9647e-03, PNorm = 69.2253, GNorm = 0.1657, lr_0 = 6.9548e-04
Validation auc = 0.715493
Epoch 220
Validation auc = 0.715962
Epoch 221
Validation auc = 0.715962
Epoch 222
Validation auc = 0.715493
Epoch 223
Validation auc = 0.718310
Epoch 224
Loss = 2.8162e-03, PNorm = 69.2911, GNorm = 0.1041, lr_0 = 6.8953e-04
Validation auc = 0.723474
Epoch 225
Validation auc = 0.724883
Epoch 226
Validation auc = 0.727700
Epoch 227
Validation auc = 0.732864
Epoch 228
Validation auc = 0.731925
Epoch 229
Loss = 3.5851e-03, PNorm = 69.3345, GNorm = 0.1880, lr_0 = 6.8364e-04
Validation auc = 0.731455
Epoch 230
Validation auc = 0.731925
Epoch 231
Validation auc = 0.731455
Epoch 232
Validation auc = 0.730986
Epoch 233
Validation auc = 0.730047
Epoch 234
Loss = 4.6124e-03, PNorm = 69.3847, GNorm = 0.1197, lr_0 = 6.7779e-04
Validation auc = 0.728638
Epoch 235
Validation auc = 0.722535
Epoch 236
Validation auc = 0.717840
Epoch 237
Validation auc = 0.713615
Epoch 238
Validation auc = 0.712207
Epoch 239
Loss = 2.5174e-03, PNorm = 69.4202, GNorm = 0.0698, lr_0 = 6.7200e-04
Validation auc = 0.713146
Epoch 240
Validation auc = 0.713615
Epoch 241
Validation auc = 0.715493
Epoch 242
Validation auc = 0.717371
Epoch 243
Validation auc = 0.718779
Epoch 244
Loss = 2.8701e-03, PNorm = 69.4423, GNorm = 0.2068, lr_0 = 6.6625e-04
Validation auc = 0.720188
Epoch 245
Validation auc = 0.718310
Epoch 246
Validation auc = 0.718779
Epoch 247
Validation auc = 0.716432
Epoch 248
Validation auc = 0.711268
Epoch 249
Loss = 1.6273e-03, PNorm = 69.4723, GNorm = 0.0388, lr_0 = 6.6056e-04
Validation auc = 0.703756
Epoch 250
Validation auc = 0.694366
Epoch 251
Validation auc = 0.693427
Epoch 252
Validation auc = 0.692488
Epoch 253
Validation auc = 0.691080
Epoch 254
Loss = 5.8348e-03, PNorm = 69.5238, GNorm = 0.2881, lr_0 = 6.5491e-04
Validation auc = 0.693427
Epoch 255
Validation auc = 0.700939
Epoch 256
Validation auc = 0.704695
Epoch 257
Validation auc = 0.706573
Epoch 258
Validation auc = 0.708920
Epoch 259
Loss = 5.5077e-03, PNorm = 69.5566, GNorm = 0.2381, lr_0 = 6.4931e-04
Validation auc = 0.710798
Epoch 260
Validation auc = 0.718310
Epoch 261
Validation auc = 0.720657
Epoch 262
Validation auc = 0.723944
Epoch 263
Validation auc = 0.720657
Epoch 264
Loss = 3.0138e-03, PNorm = 69.6027, GNorm = 0.1232, lr_0 = 6.4376e-04
Validation auc = 0.719249
Epoch 265
Validation auc = 0.719249
Epoch 266
Validation auc = 0.721127
Epoch 267
Validation auc = 0.717840
Epoch 268
Validation auc = 0.726761
Epoch 269
Loss = 3.3031e-03, PNorm = 69.6371, GNorm = 0.1770, lr_0 = 6.3826e-04
Validation auc = 0.737089
Epoch 270
Validation auc = 0.741315
Epoch 271
Validation auc = 0.742723
Epoch 272
Validation auc = 0.731455
Epoch 273
Validation auc = 0.726761
Epoch 274
Loss = 8.6551e-04, PNorm = 69.6772, GNorm = 0.0296, lr_0 = 6.3280e-04
Validation auc = 0.721596
Epoch 275
Validation auc = 0.713146
Epoch 276
Validation auc = 0.707512
Epoch 277
Validation auc = 0.703286
Epoch 278
Validation auc = 0.699531
Epoch 279
Loss = 1.2058e-03, PNorm = 69.7084, GNorm = 0.0199, lr_0 = 6.2739e-04
Validation auc = 0.700469
Epoch 280
Validation auc = 0.704695
Epoch 281
Validation auc = 0.707512
Epoch 282
Validation auc = 0.708451
Epoch 283
Validation auc = 0.708451
Epoch 284
Loss = 4.2928e-03, PNorm = 69.7374, GNorm = 0.2155, lr_0 = 6.2203e-04
Validation auc = 0.707512
Epoch 285
Validation auc = 0.709390
Epoch 286
Validation auc = 0.709859
Epoch 287
Validation auc = 0.712676
Epoch 288
Validation auc = 0.730986
Epoch 289
Loss = 1.5731e-03, PNorm = 69.7705, GNorm = 0.0633, lr_0 = 6.1671e-04
Validation auc = 0.740376
Epoch 290
Validation auc = 0.748826
Epoch 291
Validation auc = 0.737089
Epoch 292
Validation auc = 0.722066
Epoch 293
Validation auc = 0.714085
Epoch 294
Loss = 1.1115e-03, PNorm = 69.8088, GNorm = 0.0644, lr_0 = 6.1144e-04
Validation auc = 0.704225
Epoch 295
Validation auc = 0.700469
Epoch 296
Validation auc = 0.698122
Epoch 297
Validation auc = 0.695775
Epoch 298
Validation auc = 0.696714
Epoch 299
Loss = 4.2317e-03, PNorm = 69.8408, GNorm = 0.1375, lr_0 = 6.0621e-04
Validation auc = 0.700939
Model 0 best validation auc = 0.748826 on epoch 290
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.875978
Ensemble test auc = 0.875978
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.611737
Epoch 1
Validation auc = 0.702347
Epoch 2
Validation auc = 0.706103
Epoch 3
Validation auc = 0.743192
Epoch 4
Loss = 6.0335e-01, PNorm = 63.7710, GNorm = 3.9278, lr_0 = 6.5000e-04
Validation auc = 0.739906
Epoch 5
Validation auc = 0.726291
Epoch 6
Validation auc = 0.737559
Epoch 7
Validation auc = 0.738498
Epoch 8
Validation auc = 0.744601
Epoch 9
Loss = 4.4061e-01, PNorm = 63.9287, GNorm = 2.4255, lr_0 = 9.9743e-04
Validation auc = 0.730047
Epoch 10
Validation auc = 0.723944
Epoch 11
Validation auc = 0.722066
Epoch 12
Validation auc = 0.721127
Epoch 13
Validation auc = 0.723944
Epoch 14
Loss = 3.8353e-01, PNorm = 64.1030, GNorm = 1.7087, lr_0 = 9.8890e-04
Validation auc = 0.739437
Epoch 15
Validation auc = 0.733803
Epoch 16
Validation auc = 0.715493
Epoch 17
Validation auc = 0.720657
Epoch 18
Validation auc = 0.725822
Epoch 19
Loss = 3.0300e-01, PNorm = 64.2321, GNorm = 2.1964, lr_0 = 9.8045e-04
Validation auc = 0.733803
Epoch 20
Validation auc = 0.732394
Epoch 21
Validation auc = 0.723944
Epoch 22
Validation auc = 0.730516
Epoch 23
Validation auc = 0.731925
Epoch 24
Loss = 2.6649e-01, PNorm = 64.3329, GNorm = 1.9058, lr_0 = 9.7207e-04
Validation auc = 0.736150
Epoch 25
Validation auc = 0.735211
Epoch 26
Validation auc = 0.731925
Epoch 27
Validation auc = 0.737089
Epoch 28
Validation auc = 0.738967
Epoch 29
Loss = 2.8609e-01, PNorm = 64.4156, GNorm = 2.6528, lr_0 = 9.6376e-04
Validation auc = 0.732864
Epoch 30
Validation auc = 0.726761
Epoch 31
Validation auc = 0.720188
Epoch 32
Validation auc = 0.719718
Epoch 33
Validation auc = 0.728638
Epoch 34
Loss = 2.3073e-01, PNorm = 64.4986, GNorm = 1.4354, lr_0 = 9.5552e-04
Validation auc = 0.730047
Epoch 35
Validation auc = 0.727230
Epoch 36
Validation auc = 0.725352
Epoch 37
Validation auc = 0.723005
Epoch 38
Validation auc = 0.718779
Epoch 39
Loss = 1.8219e-01, PNorm = 64.5929, GNorm = 1.2446, lr_0 = 9.4735e-04
Validation auc = 0.711268
Epoch 40
Validation auc = 0.700939
Epoch 41
Validation auc = 0.709859
Epoch 42
Validation auc = 0.715023
Epoch 43
Validation auc = 0.730047
Epoch 44
Loss = 1.3944e-01, PNorm = 64.7069, GNorm = 1.4163, lr_0 = 9.3925e-04
Validation auc = 0.732864
Epoch 45
Validation auc = 0.735211
Epoch 46
Validation auc = 0.731925
Epoch 47
Validation auc = 0.726761
Epoch 48
Validation auc = 0.726761
Epoch 49
Loss = 9.6783e-02, PNorm = 64.8406, GNorm = 0.8621, lr_0 = 9.3122e-04
Validation auc = 0.723944
Epoch 50
Validation auc = 0.725822
Epoch 51
Validation auc = 0.736150
Epoch 52
Validation auc = 0.739437
Epoch 53
Validation auc = 0.736150
Epoch 54
Loss = 8.4768e-02, PNorm = 64.9991, GNorm = 0.4261, lr_0 = 9.2326e-04
Validation auc = 0.730516
Epoch 55
Validation auc = 0.720657
Epoch 56
Validation auc = 0.711737
Epoch 57
Validation auc = 0.701408
Epoch 58
Validation auc = 0.700469
Epoch 59
Loss = 2.0834e-01, PNorm = 65.1424, GNorm = 2.6691, lr_0 = 9.1537e-04
Validation auc = 0.707512
Epoch 60
Validation auc = 0.719718
Epoch 61
Validation auc = 0.718779
Epoch 62
Validation auc = 0.716901
Epoch 63
Validation auc = 0.725822
Epoch 64
Loss = 1.1955e-01, PNorm = 65.3035, GNorm = 1.2285, lr_0 = 9.0754e-04
Validation auc = 0.731455
Epoch 65
Validation auc = 0.734272
Epoch 66
Validation auc = 0.733333
Epoch 67
Validation auc = 0.732394
Epoch 68
Validation auc = 0.730516
Epoch 69
Loss = 1.1429e-01, PNorm = 65.4567, GNorm = 0.8786, lr_0 = 8.9978e-04
Validation auc = 0.732864
Epoch 70
Validation auc = 0.730986
Epoch 71
Validation auc = 0.732864
Epoch 72
Validation auc = 0.737559
Epoch 73
Validation auc = 0.737559
Epoch 74
Loss = 1.0033e-01, PNorm = 65.6121, GNorm = 0.9684, lr_0 = 8.9209e-04
Validation auc = 0.737559
Epoch 75
Validation auc = 0.732394
Epoch 76
Validation auc = 0.722535
Epoch 77
Validation auc = 0.722066
Epoch 78
Validation auc = 0.722535
Epoch 79
Loss = 4.5139e-02, PNorm = 65.7591, GNorm = 0.8750, lr_0 = 8.8447e-04
Validation auc = 0.720657
Epoch 80
Validation auc = 0.725352
Epoch 81
Validation auc = 0.720657
Epoch 82
Validation auc = 0.708920
Epoch 83
Validation auc = 0.705634
Epoch 84
Loss = 6.9890e-02, PNorm = 65.8830, GNorm = 0.7530, lr_0 = 8.7691e-04
Validation auc = 0.710798
Epoch 85
Validation auc = 0.715493
Epoch 86
Validation auc = 0.723005
Epoch 87
Validation auc = 0.730516
Epoch 88
Validation auc = 0.737089
Epoch 89
Loss = 4.9416e-02, PNorm = 66.0278, GNorm = 0.7339, lr_0 = 8.6941e-04
Validation auc = 0.742254
Epoch 90
Validation auc = 0.741784
Epoch 91
Validation auc = 0.745070
Epoch 92
Validation auc = 0.742254
Epoch 93
Validation auc = 0.726291
Epoch 94
Loss = 5.0390e-02, PNorm = 66.1666, GNorm = 0.4714, lr_0 = 8.6198e-04
Validation auc = 0.718310
Epoch 95
Validation auc = 0.707512
Epoch 96
Validation auc = 0.706103
Epoch 97
Validation auc = 0.716901
Epoch 98
Validation auc = 0.709859
Epoch 99
Loss = 3.3359e-02, PNorm = 66.3395, GNorm = 0.2681, lr_0 = 8.5461e-04
Validation auc = 0.703286
Epoch 100
Validation auc = 0.700469
Epoch 101
Validation auc = 0.705634
Epoch 102
Validation auc = 0.707981
Epoch 103
Validation auc = 0.710329
Epoch 104
Loss = 4.6236e-02, PNorm = 66.5045, GNorm = 1.0119, lr_0 = 8.4730e-04
Validation auc = 0.702817
Epoch 105
Validation auc = 0.696714
Epoch 106
Validation auc = 0.694366
Epoch 107
Validation auc = 0.692958
Epoch 108
Validation auc = 0.695305
Epoch 109
Loss = 7.5685e-02, PNorm = 66.6690, GNorm = 1.1634, lr_0 = 8.4006e-04
Validation auc = 0.701878
Epoch 110
Validation auc = 0.707981
Epoch 111
Validation auc = 0.708920
Epoch 112
Validation auc = 0.713146
Epoch 113
Validation auc = 0.711737
Epoch 114
Loss = 4.5227e-02, PNorm = 66.8260, GNorm = 0.6045, lr_0 = 8.3288e-04
Validation auc = 0.713615
Epoch 115
Validation auc = 0.716432
Epoch 116
Validation auc = 0.716432
Epoch 117
Validation auc = 0.713146
Epoch 118
Validation auc = 0.708451
Epoch 119
Loss = 3.2620e-02, PNorm = 66.9769, GNorm = 0.2408, lr_0 = 8.2576e-04
Validation auc = 0.707981
Epoch 120
Validation auc = 0.705164
Epoch 121
Validation auc = 0.703756
Epoch 122
Validation auc = 0.699531
Epoch 123
Validation auc = 0.699531
Epoch 124
Loss = 1.9166e-02, PNorm = 67.0889, GNorm = 0.0752, lr_0 = 8.1870e-04
Validation auc = 0.701878
Epoch 125
Validation auc = 0.705634
Epoch 126
Validation auc = 0.717840
Epoch 127
Validation auc = 0.723005
Epoch 128
Validation auc = 0.723944
Epoch 129
Loss = 5.0975e-02, PNorm = 67.1882, GNorm = 0.7899, lr_0 = 8.1170e-04
Validation auc = 0.712207
Epoch 130
Validation auc = 0.701408
Epoch 131
Validation auc = 0.708451
Epoch 132
Validation auc = 0.713146
Epoch 133
Validation auc = 0.710798
Epoch 134
Loss = 6.4396e-02, PNorm = 67.3254, GNorm = 0.5659, lr_0 = 8.0476e-04
Validation auc = 0.702347
Epoch 135
Validation auc = 0.698592
Epoch 136
Validation auc = 0.692958
Epoch 137
Validation auc = 0.688263
Epoch 138
Validation auc = 0.686385
Epoch 139
Loss = 5.1465e-02, PNorm = 67.4716, GNorm = 0.6589, lr_0 = 7.9788e-04
Validation auc = 0.686385
Epoch 140
Validation auc = 0.685446
Epoch 141
Validation auc = 0.688263
Epoch 142
Validation auc = 0.687324
Epoch 143
Validation auc = 0.688263
Epoch 144
Loss = 1.5349e-02, PNorm = 67.5762, GNorm = 0.1643, lr_0 = 7.9106e-04
Validation auc = 0.690141
Epoch 145
Validation auc = 0.689671
Epoch 146
Validation auc = 0.692488
Epoch 147
Validation auc = 0.692958
Epoch 148
Validation auc = 0.695305
Epoch 149
Loss = 3.8559e-02, PNorm = 67.6830, GNorm = 0.2161, lr_0 = 7.8430e-04
Validation auc = 0.689202
Epoch 150
Validation auc = 0.679812
Epoch 151
Validation auc = 0.680751
Epoch 152
Validation auc = 0.679812
Epoch 153
Validation auc = 0.684977
Epoch 154
Loss = 3.7635e-02, PNorm = 67.7974, GNorm = 0.2274, lr_0 = 7.7759e-04
Validation auc = 0.695305
Epoch 155
Validation auc = 0.703286
Epoch 156
Validation auc = 0.706573
Epoch 157
Validation auc = 0.712676
Epoch 158
Validation auc = 0.713615
Epoch 159
Loss = 6.1673e-02, PNorm = 67.9038, GNorm = 0.2857, lr_0 = 7.7095e-04
Validation auc = 0.712207
Epoch 160
Validation auc = 0.721596
Epoch 161
Validation auc = 0.717840
Epoch 162
Validation auc = 0.703286
Epoch 163
Validation auc = 0.697183
Epoch 164
Loss = 2.6941e-02, PNorm = 68.0146, GNorm = 0.4430, lr_0 = 7.6436e-04
Validation auc = 0.693427
Epoch 165
Validation auc = 0.691549
Epoch 166
Validation auc = 0.694366
Epoch 167
Validation auc = 0.700469
Epoch 168
Validation auc = 0.703286
Epoch 169
Loss = 1.7725e-02, PNorm = 68.1260, GNorm = 0.3154, lr_0 = 7.5782e-04
Validation auc = 0.702347
Epoch 170
Validation auc = 0.695775
Epoch 171
Validation auc = 0.691549
Epoch 172
Validation auc = 0.690610
Epoch 173
Validation auc = 0.688732
Epoch 174
Loss = 2.9053e-02, PNorm = 68.2131, GNorm = 0.1275, lr_0 = 7.5134e-04
Validation auc = 0.697183
Epoch 175
Validation auc = 0.699061
Epoch 176
Validation auc = 0.701408
Epoch 177
Validation auc = 0.699061
Epoch 178
Validation auc = 0.694366
Epoch 179
Loss = 1.2104e-02, PNorm = 68.3101, GNorm = 0.0965, lr_0 = 7.4492e-04
Validation auc = 0.694836
Epoch 180
Validation auc = 0.700939
Epoch 181
Validation auc = 0.705634
Epoch 182
Validation auc = 0.706103
Epoch 183
Validation auc = 0.707981
Epoch 184
Loss = 2.1037e-02, PNorm = 68.4257, GNorm = 0.5018, lr_0 = 7.3855e-04
Validation auc = 0.709390
Epoch 185
Validation auc = 0.709390
Epoch 186
Validation auc = 0.711737
Epoch 187
Validation auc = 0.717371
Epoch 188
Validation auc = 0.718779
Epoch 189
Loss = 5.0500e-02, PNorm = 68.5353, GNorm = 0.3652, lr_0 = 7.3224e-04
Validation auc = 0.719249
Epoch 190
Validation auc = 0.723944
Epoch 191
Validation auc = 0.725822
Epoch 192
Validation auc = 0.725352
Epoch 193
Validation auc = 0.725352
Epoch 194
Loss = 2.8464e-02, PNorm = 68.6234, GNorm = 0.4792, lr_0 = 7.2598e-04
Validation auc = 0.728638
Epoch 195
Validation auc = 0.725822
Epoch 196
Validation auc = 0.727230
Epoch 197
Validation auc = 0.723005
Epoch 198
Validation auc = 0.716901
Epoch 199
Loss = 8.9497e-03, PNorm = 68.7260, GNorm = 0.2044, lr_0 = 7.1977e-04
Validation auc = 0.707042
Epoch 200
Validation auc = 0.699531
Epoch 201
Validation auc = 0.716432
Epoch 202
Validation auc = 0.720188
Epoch 203
Validation auc = 0.725352
Epoch 204
Loss = 7.3188e-03, PNorm = 68.7982, GNorm = 0.2099, lr_0 = 7.1362e-04
Validation auc = 0.728169
Epoch 205
Validation auc = 0.732864
Epoch 206
Validation auc = 0.734742
Epoch 207
Validation auc = 0.733333
Epoch 208
Validation auc = 0.734742
Epoch 209
Loss = 1.9411e-02, PNorm = 68.8911, GNorm = 0.4216, lr_0 = 7.0752e-04
Validation auc = 0.735681
Epoch 210
Validation auc = 0.733333
Epoch 211
Validation auc = 0.733333
Epoch 212
Validation auc = 0.729108
Epoch 213
Validation auc = 0.730516
Epoch 214
Loss = 6.9376e-03, PNorm = 69.0007, GNorm = 0.0738, lr_0 = 7.0147e-04
Validation auc = 0.725352
Epoch 215
Validation auc = 0.724883
Epoch 216
Validation auc = 0.721596
Epoch 217
Validation auc = 0.723944
Epoch 218
Validation auc = 0.730516
Epoch 219
Loss = 1.7964e-02, PNorm = 69.0691, GNorm = 0.4510, lr_0 = 6.9548e-04
Validation auc = 0.734272
Epoch 220
Validation auc = 0.737559
Epoch 221
Validation auc = 0.738028
Epoch 222
Validation auc = 0.734272
Epoch 223
Validation auc = 0.732394
Epoch 224
Loss = 6.9203e-03, PNorm = 69.1231, GNorm = 0.4582, lr_0 = 6.8953e-04
Validation auc = 0.731925
Epoch 225
Validation auc = 0.733333
Epoch 226
Validation auc = 0.740376
Epoch 227
Validation auc = 0.744131
Epoch 228
Validation auc = 0.746009
Epoch 229
Loss = 1.6932e-02, PNorm = 69.1876, GNorm = 0.2380, lr_0 = 6.8364e-04
Validation auc = 0.744131
Epoch 230
Validation auc = 0.738498
Epoch 231
Validation auc = 0.731925
Epoch 232
Validation auc = 0.726291
Epoch 233
Validation auc = 0.723005
Epoch 234
Loss = 2.9948e-03, PNorm = 69.2524, GNorm = 0.0772, lr_0 = 6.7779e-04
Validation auc = 0.721596
Epoch 235
Validation auc = 0.720188
Epoch 236
Validation auc = 0.718310
Epoch 237
Validation auc = 0.715962
Epoch 238
Validation auc = 0.715023
Epoch 239
Loss = 2.4399e-02, PNorm = 69.2891, GNorm = 0.1822, lr_0 = 6.7200e-04
Validation auc = 0.710798
Epoch 240
Validation auc = 0.705634
Epoch 241
Validation auc = 0.702347
Epoch 242
Validation auc = 0.704695
Epoch 243
Validation auc = 0.707981
Epoch 244
Loss = 4.2395e-03, PNorm = 69.3448, GNorm = 0.1937, lr_0 = 6.6625e-04
Validation auc = 0.707981
Epoch 245
Validation auc = 0.708920
Epoch 246
Validation auc = 0.710798
Epoch 247
Validation auc = 0.709859
Epoch 248
Validation auc = 0.708451
Epoch 249
Loss = 2.0000e-02, PNorm = 69.4244, GNorm = 0.4639, lr_0 = 6.6056e-04
Validation auc = 0.705164
Epoch 250
Validation auc = 0.707042
Epoch 251
Validation auc = 0.707042
Epoch 252
Validation auc = 0.710798
Epoch 253
Validation auc = 0.732394
Epoch 254
Loss = 7.0548e-03, PNorm = 69.5171, GNorm = 0.1292, lr_0 = 6.5491e-04
Validation auc = 0.759155
Epoch 255
Validation auc = 0.744131
Epoch 256
Validation auc = 0.734742
Epoch 257
Validation auc = 0.730047
Epoch 258
Validation auc = 0.730047
Epoch 259
Loss = 8.3275e-03, PNorm = 69.6028, GNorm = 0.0883, lr_0 = 6.4931e-04
Validation auc = 0.734272
Epoch 260
Validation auc = 0.738028
Epoch 261
Validation auc = 0.737559
Epoch 262
Validation auc = 0.737089
Epoch 263
Validation auc = 0.733333
Epoch 264
Loss = 9.3902e-03, PNorm = 69.6667, GNorm = 0.5411, lr_0 = 6.4376e-04
Validation auc = 0.730986
Epoch 265
Validation auc = 0.729108
Epoch 266
Validation auc = 0.727700
Epoch 267
Validation auc = 0.729577
Epoch 268
Validation auc = 0.727700
Epoch 269
Loss = 1.0979e-02, PNorm = 69.7218, GNorm = 0.1612, lr_0 = 6.3826e-04
Validation auc = 0.723944
Epoch 270
Validation auc = 0.723474
Epoch 271
Validation auc = 0.721127
Epoch 272
Validation auc = 0.721127
Epoch 273
Validation auc = 0.720657
Epoch 274
Loss = 4.1325e-03, PNorm = 69.7712, GNorm = 0.2417, lr_0 = 6.3280e-04
Validation auc = 0.722535
Epoch 275
Validation auc = 0.723005
Epoch 276
Validation auc = 0.722535
Epoch 277
Validation auc = 0.723005
Epoch 278
Validation auc = 0.721596
Epoch 279
Loss = 9.9026e-03, PNorm = 69.8082, GNorm = 0.0186, lr_0 = 6.2739e-04
Validation auc = 0.723005
Epoch 280
Validation auc = 0.722066
Epoch 281
Validation auc = 0.720657
Epoch 282
Validation auc = 0.717371
Epoch 283
Validation auc = 0.716432
Epoch 284
Loss = 3.8253e-03, PNorm = 69.8530, GNorm = 0.0783, lr_0 = 6.2203e-04
Validation auc = 0.711737
Epoch 285
Validation auc = 0.707042
Epoch 286
Validation auc = 0.702817
Epoch 287
Validation auc = 0.711268
Epoch 288
Validation auc = 0.715493
Epoch 289
Loss = 1.9937e-03, PNorm = 69.8916, GNorm = 0.0350, lr_0 = 6.1671e-04
Validation auc = 0.720188
Epoch 290
Validation auc = 0.722066
Epoch 291
Validation auc = 0.723005
Epoch 292
Validation auc = 0.722535
Epoch 293
Validation auc = 0.719249
Epoch 294
Loss = 9.0417e-04, PNorm = 69.9378, GNorm = 0.0242, lr_0 = 6.1144e-04
Validation auc = 0.715023
Epoch 295
Validation auc = 0.715962
Epoch 296
Validation auc = 0.714085
Epoch 297
Validation auc = 0.706103
Epoch 298
Validation auc = 0.700000
Epoch 299
Loss = 3.2277e-03, PNorm = 69.9858, GNorm = 0.2756, lr_0 = 6.0621e-04
Validation auc = 0.699061
Model 0 best validation auc = 0.759155 on epoch 254
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.919326
Ensemble test auc = 0.919326
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.666667
Epoch 1
Validation auc = 0.688732
Epoch 2
Validation auc = 0.692019
Epoch 3
Validation auc = 0.706573
Epoch 4
Loss = 5.2938e-01, PNorm = 63.7795, GNorm = 2.1256, lr_0 = 6.5000e-04
Validation auc = 0.712676
Epoch 5
Validation auc = 0.729108
Epoch 6
Validation auc = 0.727700
Epoch 7
Validation auc = 0.747418
Epoch 8
Validation auc = 0.756338
Epoch 9
Loss = 4.1725e-01, PNorm = 63.9519, GNorm = 3.1969, lr_0 = 9.9743e-04
Validation auc = 0.769484
Epoch 10
Validation auc = 0.771362
Epoch 11
Validation auc = 0.756338
Epoch 12
Validation auc = 0.752113
Epoch 13
Validation auc = 0.747418
Epoch 14
Loss = 3.3277e-01, PNorm = 64.1347, GNorm = 1.2982, lr_0 = 9.8890e-04
Validation auc = 0.725352
Epoch 15
Validation auc = 0.712207
Epoch 16
Validation auc = 0.708451
Epoch 17
Validation auc = 0.704225
Epoch 18
Validation auc = 0.700469
Epoch 19
Loss = 2.8970e-01, PNorm = 64.2791, GNorm = 1.3492, lr_0 = 9.8045e-04
Validation auc = 0.701408
Epoch 20
Validation auc = 0.701878
Epoch 21
Validation auc = 0.714085
Epoch 22
Validation auc = 0.723005
Epoch 23
Validation auc = 0.723944
Epoch 24
Loss = 3.1399e-01, PNorm = 64.3870, GNorm = 1.9302, lr_0 = 9.7207e-04
Validation auc = 0.721596
Epoch 25
Validation auc = 0.720657
Epoch 26
Validation auc = 0.714085
Epoch 27
Validation auc = 0.707042
Epoch 28
Validation auc = 0.700000
Epoch 29
Loss = 2.1442e-01, PNorm = 64.4832, GNorm = 2.0479, lr_0 = 9.6376e-04
Validation auc = 0.709390
Epoch 30
Validation auc = 0.719718
Epoch 31
Validation auc = 0.725822
Epoch 32
Validation auc = 0.721127
Epoch 33
Validation auc = 0.713146
Epoch 34
Loss = 1.2678e-01, PNorm = 64.5958, GNorm = 1.0804, lr_0 = 9.5552e-04
Validation auc = 0.707042
Epoch 35
Validation auc = 0.700000
Epoch 36
Validation auc = 0.702347
Epoch 37
Validation auc = 0.702817
Epoch 38
Validation auc = 0.706573
Epoch 39
Loss = 2.8590e-01, PNorm = 64.7210, GNorm = 1.8579, lr_0 = 9.4735e-04
Validation auc = 0.717371
Epoch 40
Validation auc = 0.718779
Epoch 41
Validation auc = 0.721127
Epoch 42
Validation auc = 0.724883
Epoch 43
Validation auc = 0.716432
Epoch 44
Loss = 1.3961e-01, PNorm = 64.8465, GNorm = 1.0495, lr_0 = 9.3925e-04
Validation auc = 0.703756
Epoch 45
Validation auc = 0.697653
Epoch 46
Validation auc = 0.701878
Epoch 47
Validation auc = 0.707981
Epoch 48
Validation auc = 0.703286
Epoch 49
Loss = 1.1200e-01, PNorm = 64.9896, GNorm = 0.7777, lr_0 = 9.3122e-04
Validation auc = 0.703286
Epoch 50
Validation auc = 0.715493
Epoch 51
Validation auc = 0.714554
Epoch 52
Validation auc = 0.713146
Epoch 53
Validation auc = 0.716901
Epoch 54
Loss = 1.3230e-01, PNorm = 65.1320, GNorm = 0.8914, lr_0 = 9.2326e-04
Validation auc = 0.718779
Epoch 55
Validation auc = 0.729108
Epoch 56
Validation auc = 0.722535
Epoch 57
Validation auc = 0.707512
Epoch 58
Validation auc = 0.707981
Epoch 59
Loss = 8.4883e-02, PNorm = 65.2862, GNorm = 1.0498, lr_0 = 9.1537e-04
Validation auc = 0.709859
Epoch 60
Validation auc = 0.709390
Epoch 61
Validation auc = 0.706573
Epoch 62
Validation auc = 0.707512
Epoch 63
Validation auc = 0.719249
Epoch 64
Loss = 9.4572e-02, PNorm = 65.4525, GNorm = 1.2971, lr_0 = 9.0754e-04
Validation auc = 0.723005
Epoch 65
Validation auc = 0.723944
Epoch 66
Validation auc = 0.717840
Epoch 67
Validation auc = 0.715023
Epoch 68
Validation auc = 0.719249
Epoch 69
Loss = 1.5810e-01, PNorm = 65.5928, GNorm = 1.7991, lr_0 = 8.9978e-04
Validation auc = 0.723005
Epoch 70
Validation auc = 0.716901
Epoch 71
Validation auc = 0.710329
Epoch 72
Validation auc = 0.706103
Epoch 73
Validation auc = 0.711268
Epoch 74
Loss = 6.3485e-02, PNorm = 65.7373, GNorm = 0.8659, lr_0 = 8.9209e-04
Validation auc = 0.714554
Epoch 75
Validation auc = 0.715023
Epoch 76
Validation auc = 0.710329
Epoch 77
Validation auc = 0.711268
Epoch 78
Validation auc = 0.705164
Epoch 79
Loss = 5.7820e-02, PNorm = 65.8724, GNorm = 0.5598, lr_0 = 8.8447e-04
Validation auc = 0.707981
Epoch 80
Validation auc = 0.709390
Epoch 81
Validation auc = 0.707512
Epoch 82
Validation auc = 0.704695
Epoch 83
Validation auc = 0.704695
Epoch 84
Loss = 8.7520e-02, PNorm = 65.9955, GNorm = 0.9533, lr_0 = 8.7691e-04
Validation auc = 0.690141
Epoch 85
Validation auc = 0.684038
Epoch 86
Validation auc = 0.687793
Epoch 87
Validation auc = 0.694836
Epoch 88
Validation auc = 0.703286
Epoch 89
Loss = 6.6598e-02, PNorm = 66.1391, GNorm = 1.2945, lr_0 = 8.6941e-04
Validation auc = 0.714085
Epoch 90
Validation auc = 0.722066
Epoch 91
Validation auc = 0.727230
Epoch 92
Validation auc = 0.725822
Epoch 93
Validation auc = 0.712676
Epoch 94
Loss = 1.2705e-01, PNorm = 66.2887, GNorm = 1.2824, lr_0 = 8.6198e-04
Validation auc = 0.703756
Epoch 95
Validation auc = 0.699531
Epoch 96
Validation auc = 0.698592
Epoch 97
Validation auc = 0.696714
Epoch 98
Validation auc = 0.691080
Epoch 99
Loss = 1.1191e-01, PNorm = 66.4452, GNorm = 0.8764, lr_0 = 8.5461e-04
Validation auc = 0.687324
Epoch 100
Validation auc = 0.679812
Epoch 101
Validation auc = 0.679812
Epoch 102
Validation auc = 0.681221
Epoch 103
Validation auc = 0.682629
Epoch 104
Loss = 1.0101e-01, PNorm = 66.5758, GNorm = 1.5364, lr_0 = 8.4730e-04
Validation auc = 0.684507
Epoch 105
Validation auc = 0.690141
Epoch 106
Validation auc = 0.691080
Epoch 107
Validation auc = 0.690141
Epoch 108
Validation auc = 0.687793
Epoch 109
Loss = 3.6527e-02, PNorm = 66.6959, GNorm = 0.7147, lr_0 = 8.4006e-04
Validation auc = 0.682629
Epoch 110
Validation auc = 0.692488
Epoch 111
Validation auc = 0.698592
Epoch 112
Validation auc = 0.687793
Epoch 113
Validation auc = 0.686385
Epoch 114
Loss = 4.1674e-02, PNorm = 66.8318, GNorm = 0.5699, lr_0 = 8.3288e-04
Validation auc = 0.687793
Epoch 115
Validation auc = 0.690141
Epoch 116
Validation auc = 0.695775
Epoch 117
Validation auc = 0.700000
Epoch 118
Validation auc = 0.700939
Epoch 119
Loss = 2.7472e-02, PNorm = 66.9744, GNorm = 0.4049, lr_0 = 8.2576e-04
Validation auc = 0.689202
Epoch 120
Validation auc = 0.687793
Epoch 121
Validation auc = 0.689671
Epoch 122
Validation auc = 0.687324
Epoch 123
Validation auc = 0.684507
Epoch 124
Loss = 2.4978e-02, PNorm = 67.0997, GNorm = 0.1059, lr_0 = 8.1870e-04
Validation auc = 0.684038
Epoch 125
Validation auc = 0.683099
Epoch 126
Validation auc = 0.682160
Epoch 127
Validation auc = 0.683099
Epoch 128
Validation auc = 0.684038
Epoch 129
Loss = 2.1844e-02, PNorm = 67.1998, GNorm = 0.4674, lr_0 = 8.1170e-04
Validation auc = 0.685915
Epoch 130
Validation auc = 0.683099
Epoch 131
Validation auc = 0.683568
Epoch 132
Validation auc = 0.684507
Epoch 133
Validation auc = 0.686385
Epoch 134
Loss = 4.8757e-02, PNorm = 67.3446, GNorm = 0.6686, lr_0 = 8.0476e-04
Validation auc = 0.692958
Epoch 135
Validation auc = 0.701408
Epoch 136
Validation auc = 0.708451
Epoch 137
Validation auc = 0.699531
Epoch 138
Validation auc = 0.691549
Epoch 139
Loss = 6.2368e-02, PNorm = 67.4733, GNorm = 0.7288, lr_0 = 7.9788e-04
Validation auc = 0.687324
Epoch 140
Validation auc = 0.680751
Epoch 141
Validation auc = 0.669953
Epoch 142
Validation auc = 0.661972
Epoch 143
Validation auc = 0.682160
Epoch 144
Loss = 2.8035e-02, PNorm = 67.6098, GNorm = 0.8201, lr_0 = 7.9106e-04
Validation auc = 0.697183
Epoch 145
Validation auc = 0.676526
Epoch 146
Validation auc = 0.668545
Epoch 147
Validation auc = 0.671362
Epoch 148
Validation auc = 0.686385
Epoch 149
Loss = 5.7084e-02, PNorm = 67.7743, GNorm = 0.5980, lr_0 = 7.8430e-04
Validation auc = 0.703286
Epoch 150
Validation auc = 0.710798
Epoch 151
Validation auc = 0.705164
Epoch 152
Validation auc = 0.693897
Epoch 153
Validation auc = 0.684507
Epoch 154
Loss = 6.0068e-02, PNorm = 67.9759, GNorm = 1.0249, lr_0 = 7.7759e-04
Validation auc = 0.690610
Epoch 155
Validation auc = 0.702817
Epoch 156
Validation auc = 0.715962
Epoch 157
Validation auc = 0.714554
Epoch 158
Validation auc = 0.708451
Epoch 159
Loss = 1.0261e-01, PNorm = 68.0953, GNorm = 0.5941, lr_0 = 7.7095e-04
Validation auc = 0.701878
Epoch 160
Validation auc = 0.696714
Epoch 161
Validation auc = 0.691549
Epoch 162
Validation auc = 0.686385
Epoch 163
Validation auc = 0.686854
Epoch 164
Loss = 2.0839e-02, PNorm = 68.2229, GNorm = 0.2933, lr_0 = 7.6436e-04
Validation auc = 0.685446
Epoch 165
Validation auc = 0.682160
Epoch 166
Validation auc = 0.686385
Epoch 167
Validation auc = 0.690610
Epoch 168
Validation auc = 0.692019
Epoch 169
Loss = 1.0190e-02, PNorm = 68.3256, GNorm = 0.2953, lr_0 = 7.5782e-04
Validation auc = 0.695305
Epoch 170
Validation auc = 0.698122
Epoch 171
Validation auc = 0.698592
Epoch 172
Validation auc = 0.701878
Epoch 173
Validation auc = 0.703756
Epoch 174
Loss = 3.6746e-02, PNorm = 68.3899, GNorm = 1.0742, lr_0 = 7.5134e-04
Validation auc = 0.699531
Epoch 175
Validation auc = 0.700000
Epoch 176
Validation auc = 0.696714
Epoch 177
Validation auc = 0.694836
Epoch 178
Validation auc = 0.698592
Epoch 179
Loss = 5.6868e-03, PNorm = 68.4568, GNorm = 0.0702, lr_0 = 7.4492e-04
Validation auc = 0.697183
Epoch 180
Validation auc = 0.700000
Epoch 181
Validation auc = 0.701408
Epoch 182
Validation auc = 0.700000
Epoch 183
Validation auc = 0.694366
Epoch 184
Loss = 2.0883e-02, PNorm = 68.5455, GNorm = 0.4361, lr_0 = 7.3855e-04
Validation auc = 0.692488
Epoch 185
Validation auc = 0.690610
Epoch 186
Validation auc = 0.695305
Epoch 187
Validation auc = 0.696244
Epoch 188
Validation auc = 0.704695
Epoch 189
Loss = 2.9295e-02, PNorm = 68.6322, GNorm = 0.6343, lr_0 = 7.3224e-04
Validation auc = 0.709859
Epoch 190
Validation auc = 0.714554
Epoch 191
Validation auc = 0.715023
Epoch 192
Validation auc = 0.713146
Epoch 193
Validation auc = 0.706573
Epoch 194
Loss = 1.6386e-02, PNorm = 68.7407, GNorm = 0.4043, lr_0 = 7.2598e-04
Validation auc = 0.703286
Epoch 195
Validation auc = 0.703286
Epoch 196
Validation auc = 0.701878
Epoch 197
Validation auc = 0.704695
Epoch 198
Validation auc = 0.701408
Epoch 199
Loss = 2.2527e-02, PNorm = 68.8386, GNorm = 1.0395, lr_0 = 7.1977e-04
Validation auc = 0.700469
Epoch 200
Validation auc = 0.705634
Epoch 201
Validation auc = 0.709859
Epoch 202
Validation auc = 0.707981
Epoch 203
Validation auc = 0.704695
Epoch 204
Loss = 5.6200e-03, PNorm = 68.9158, GNorm = 0.0525, lr_0 = 7.1362e-04
Validation auc = 0.699061
Epoch 205
Validation auc = 0.695775
Epoch 206
Validation auc = 0.700939
Epoch 207
Validation auc = 0.700469
Epoch 208
Validation auc = 0.701878
Epoch 209
Loss = 2.7625e-02, PNorm = 69.0063, GNorm = 1.0505, lr_0 = 7.0752e-04
Validation auc = 0.704225
Epoch 210
Validation auc = 0.703756
Epoch 211
Validation auc = 0.699061
Epoch 212
Validation auc = 0.700000
Epoch 213
Validation auc = 0.708920
Epoch 214
Loss = 9.4613e-03, PNorm = 69.0929, GNorm = 0.2360, lr_0 = 7.0147e-04
Validation auc = 0.711268
Epoch 215
Validation auc = 0.707042
Epoch 216
Validation auc = 0.701408
Epoch 217
Validation auc = 0.696714
Epoch 218
Validation auc = 0.692958
Epoch 219
Loss = 4.3850e-03, PNorm = 69.1450, GNorm = 0.0468, lr_0 = 6.9548e-04
Validation auc = 0.690610
Epoch 220
Validation auc = 0.689671
Epoch 221
Validation auc = 0.690610
Epoch 222
Validation auc = 0.691080
Epoch 223
Validation auc = 0.690141
Epoch 224
Loss = 8.5110e-03, PNorm = 69.2016, GNorm = 0.1088, lr_0 = 6.8953e-04
Validation auc = 0.685915
Epoch 225
Validation auc = 0.684507
Epoch 226
Validation auc = 0.684507
Epoch 227
Validation auc = 0.681690
Epoch 228
Validation auc = 0.683568
Epoch 229
Loss = 1.4429e-02, PNorm = 69.2725, GNorm = 0.5766, lr_0 = 6.8364e-04
Validation auc = 0.685446
Epoch 230
Validation auc = 0.689671
Epoch 231
Validation auc = 0.691549
Epoch 232
Validation auc = 0.696714
Epoch 233
Validation auc = 0.697653
Epoch 234
Loss = 1.2265e-02, PNorm = 69.3374, GNorm = 0.3506, lr_0 = 6.7779e-04
Validation auc = 0.696244
Epoch 235
Validation auc = 0.695775
Epoch 236
Validation auc = 0.695305
Epoch 237
Validation auc = 0.697183
Epoch 238
Validation auc = 0.695775
Epoch 239
Loss = 7.2060e-03, PNorm = 69.4001, GNorm = 0.3530, lr_0 = 6.7200e-04
Validation auc = 0.692958
Epoch 240
Validation auc = 0.692019
Epoch 241
Validation auc = 0.692958
Epoch 242
Validation auc = 0.693897
Epoch 243
Validation auc = 0.694366
Epoch 244
Loss = 1.5838e-02, PNorm = 69.4572, GNorm = 0.6291, lr_0 = 6.6625e-04
Validation auc = 0.686854
Epoch 245
Validation auc = 0.685915
Epoch 246
Validation auc = 0.687324
Epoch 247
Validation auc = 0.692488
Epoch 248
Validation auc = 0.699061
Epoch 249
Loss = 1.0227e-02, PNorm = 69.5038, GNorm = 0.2491, lr_0 = 6.6056e-04
Validation auc = 0.702347
Epoch 250
Validation auc = 0.704225
Epoch 251
Validation auc = 0.703756
Epoch 252
Validation auc = 0.702817
Epoch 253
Validation auc = 0.700939
Epoch 254
Loss = 3.5949e-03, PNorm = 69.5800, GNorm = 0.0947, lr_0 = 6.5491e-04
Validation auc = 0.700939
Epoch 255
Validation auc = 0.699061
Epoch 256
Validation auc = 0.699061
Epoch 257
Validation auc = 0.692958
Epoch 258
Validation auc = 0.692488
Epoch 259
Loss = 3.1671e-03, PNorm = 69.6673, GNorm = 0.2122, lr_0 = 6.4931e-04
Validation auc = 0.690141
Epoch 260
Validation auc = 0.688732
Epoch 261
Validation auc = 0.693427
Epoch 262
Validation auc = 0.700939
Epoch 263
Validation auc = 0.707512
Epoch 264
Loss = 3.6332e-02, PNorm = 69.7150, GNorm = 2.3861, lr_0 = 6.4376e-04
Validation auc = 0.712676
Epoch 265
Validation auc = 0.710329
Epoch 266
Validation auc = 0.701878
Epoch 267
Validation auc = 0.695305
Epoch 268
Validation auc = 0.693427
Epoch 269
Loss = 6.3679e-03, PNorm = 69.8193, GNorm = 0.4123, lr_0 = 6.3826e-04
Validation auc = 0.698122
Epoch 270
Validation auc = 0.710798
Epoch 271
Validation auc = 0.717371
Epoch 272
Validation auc = 0.719249
Epoch 273
Validation auc = 0.716901
Epoch 274
Loss = 3.1422e-02, PNorm = 69.8899, GNorm = 0.8984, lr_0 = 6.3280e-04
Validation auc = 0.720657
Epoch 275
Validation auc = 0.720188
Epoch 276
Validation auc = 0.712676
Epoch 277
Validation auc = 0.713146
Epoch 278
Validation auc = 0.711268
Epoch 279
Loss = 1.6928e-03, PNorm = 69.9567, GNorm = 0.0902, lr_0 = 6.2739e-04
Validation auc = 0.709859
Epoch 280
Validation auc = 0.708451
Epoch 281
Validation auc = 0.709390
Epoch 282
Validation auc = 0.708451
Epoch 283
Validation auc = 0.706103
Epoch 284
Loss = 8.7149e-03, PNorm = 70.0182, GNorm = 0.3286, lr_0 = 6.2203e-04
Validation auc = 0.703286
Epoch 285
Validation auc = 0.703756
Epoch 286
Validation auc = 0.700939
Epoch 287
Validation auc = 0.704695
Epoch 288
Validation auc = 0.702817
Epoch 289
Loss = 5.3601e-03, PNorm = 70.0755, GNorm = 0.4213, lr_0 = 6.1671e-04
Validation auc = 0.702347
Epoch 290
Validation auc = 0.705634
Epoch 291
Validation auc = 0.704695
Epoch 292
Validation auc = 0.704695
Epoch 293
Validation auc = 0.708451
Epoch 294
Loss = 2.2973e-03, PNorm = 70.1219, GNorm = 0.0126, lr_0 = 6.1144e-04
Validation auc = 0.709859
Epoch 295
Validation auc = 0.712676
Epoch 296
Validation auc = 0.707981
Epoch 297
Validation auc = 0.704695
Epoch 298
Validation auc = 0.699531
Epoch 299
Loss = 2.0906e-03, PNorm = 70.1536, GNorm = 0.0312, lr_0 = 6.0621e-04
Validation auc = 0.697183
Model 0 best validation auc = 0.771362 on epoch 10
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.904877
Ensemble test auc = 0.904877
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.645070
Epoch 1
Validation auc = 0.695775
Epoch 2
Validation auc = 0.712676
Epoch 3
Validation auc = 0.756338
Epoch 4
Loss = 5.5199e-01, PNorm = 63.7734, GNorm = 2.3555, lr_0 = 6.5000e-04
Validation auc = 0.759624
Epoch 5
Validation auc = 0.768545
Epoch 6
Validation auc = 0.766667
Epoch 7
Validation auc = 0.761033
Epoch 8
Validation auc = 0.759624
Epoch 9
Loss = 3.8491e-01, PNorm = 63.9278, GNorm = 1.6771, lr_0 = 9.9743e-04
Validation auc = 0.726761
Epoch 10
Validation auc = 0.726291
Epoch 11
Validation auc = 0.718310
Epoch 12
Validation auc = 0.711268
Epoch 13
Validation auc = 0.714085
Epoch 14
Loss = 3.6948e-01, PNorm = 64.1035, GNorm = 1.8098, lr_0 = 9.8890e-04
Validation auc = 0.713146
Epoch 15
Validation auc = 0.716901
Epoch 16
Validation auc = 0.726761
Epoch 17
Validation auc = 0.738498
Epoch 18
Validation auc = 0.742254
Epoch 19
Loss = 3.3094e-01, PNorm = 64.2340, GNorm = 1.6147, lr_0 = 9.8045e-04
Validation auc = 0.742723
Epoch 20
Validation auc = 0.736150
Epoch 21
Validation auc = 0.737559
Epoch 22
Validation auc = 0.739437
Epoch 23
Validation auc = 0.735681
Epoch 24
Loss = 2.5985e-01, PNorm = 64.3409, GNorm = 2.7640, lr_0 = 9.7207e-04
Validation auc = 0.726761
Epoch 25
Validation auc = 0.721127
Epoch 26
Validation auc = 0.720657
Epoch 27
Validation auc = 0.727700
Epoch 28
Validation auc = 0.731455
Epoch 29
Loss = 2.4195e-01, PNorm = 64.4413, GNorm = 2.4430, lr_0 = 9.6376e-04
Validation auc = 0.729108
Epoch 30
Validation auc = 0.728638
Epoch 31
Validation auc = 0.732864
Epoch 32
Validation auc = 0.731925
Epoch 33
Validation auc = 0.730047
Epoch 34
Loss = 1.7336e-01, PNorm = 64.5372, GNorm = 1.1437, lr_0 = 9.5552e-04
Validation auc = 0.723944
Epoch 35
Validation auc = 0.726761
Epoch 36
Validation auc = 0.726761
Epoch 37
Validation auc = 0.725822
Epoch 38
Validation auc = 0.723474
Epoch 39
Loss = 2.2360e-01, PNorm = 64.6456, GNorm = 1.4512, lr_0 = 9.4735e-04
Validation auc = 0.725822
Epoch 40
Validation auc = 0.715023
Epoch 41
Validation auc = 0.708451
Epoch 42
Validation auc = 0.712676
Epoch 43
Validation auc = 0.712676
Epoch 44
Loss = 1.1082e-01, PNorm = 64.7555, GNorm = 0.8193, lr_0 = 9.3925e-04
Validation auc = 0.708451
Epoch 45
Validation auc = 0.707981
Epoch 46
Validation auc = 0.720188
Epoch 47
Validation auc = 0.731455
Epoch 48
Validation auc = 0.723944
Epoch 49
Loss = 1.4501e-01, PNorm = 64.8775, GNorm = 1.5526, lr_0 = 9.3122e-04
Validation auc = 0.723005
Epoch 50
Validation auc = 0.719249
Epoch 51
Validation auc = 0.720188
Epoch 52
Validation auc = 0.715962
Epoch 53
Validation auc = 0.703286
Epoch 54
Loss = 1.8378e-01, PNorm = 65.0226, GNorm = 2.3631, lr_0 = 9.2326e-04
Validation auc = 0.696244
Epoch 55
Validation auc = 0.713615
Epoch 56
Validation auc = 0.713146
Epoch 57
Validation auc = 0.706573
Epoch 58
Validation auc = 0.703286
Epoch 59
Loss = 1.1325e-01, PNorm = 65.1830, GNorm = 1.0899, lr_0 = 9.1537e-04
Validation auc = 0.706573
Epoch 60
Validation auc = 0.708451
Epoch 61
Validation auc = 0.720657
Epoch 62
Validation auc = 0.720657
Epoch 63
Validation auc = 0.720188
Epoch 64
Loss = 1.5674e-01, PNorm = 65.3310, GNorm = 1.8675, lr_0 = 9.0754e-04
Validation auc = 0.717840
Epoch 65
Validation auc = 0.720188
Epoch 66
Validation auc = 0.725352
Epoch 67
Validation auc = 0.724413
Epoch 68
Validation auc = 0.720188
Epoch 69
Loss = 1.4138e-01, PNorm = 65.4837, GNorm = 1.8104, lr_0 = 8.9978e-04
Validation auc = 0.716901
Epoch 70
Validation auc = 0.719249
Epoch 71
Validation auc = 0.720188
Epoch 72
Validation auc = 0.715493
Epoch 73
Validation auc = 0.712207
Epoch 74
Loss = 7.2627e-02, PNorm = 65.6445, GNorm = 0.6815, lr_0 = 8.9209e-04
Validation auc = 0.709390
Epoch 75
Validation auc = 0.712207
Epoch 76
Validation auc = 0.709859
Epoch 77
Validation auc = 0.714554
Epoch 78
Validation auc = 0.716432
Epoch 79
Loss = 4.9061e-02, PNorm = 65.7864, GNorm = 0.7382, lr_0 = 8.8447e-04
Validation auc = 0.707981
Epoch 80
Validation auc = 0.715023
Epoch 81
Validation auc = 0.720657
Epoch 82
Validation auc = 0.721127
Epoch 83
Validation auc = 0.718779
Epoch 84
Loss = 4.7776e-02, PNorm = 65.9463, GNorm = 0.3446, lr_0 = 8.7691e-04
Validation auc = 0.708920
Epoch 85
Validation auc = 0.721596
Epoch 86
Validation auc = 0.725822
Epoch 87
Validation auc = 0.723005
Epoch 88
Validation auc = 0.724883
Epoch 89
Loss = 7.9526e-02, PNorm = 66.0863, GNorm = 1.4356, lr_0 = 8.6941e-04
Validation auc = 0.731925
Epoch 90
Validation auc = 0.736150
Epoch 91
Validation auc = 0.734272
Epoch 92
Validation auc = 0.728169
Epoch 93
Validation auc = 0.721596
Epoch 94
Loss = 5.6382e-02, PNorm = 66.2399, GNorm = 1.1241, lr_0 = 8.6198e-04
Validation auc = 0.708920
Epoch 95
Validation auc = 0.711737
Epoch 96
Validation auc = 0.712207
Epoch 97
Validation auc = 0.711737
Epoch 98
Validation auc = 0.711268
Epoch 99
Loss = 6.5329e-02, PNorm = 66.3668, GNorm = 1.5613, lr_0 = 8.5461e-04
Validation auc = 0.709390
Epoch 100
Validation auc = 0.716432
Epoch 101
Validation auc = 0.727700
Epoch 102
Validation auc = 0.721127
Epoch 103
Validation auc = 0.710329
Epoch 104
Loss = 4.0148e-02, PNorm = 66.5058, GNorm = 0.4646, lr_0 = 8.4730e-04
Validation auc = 0.708920
Epoch 105
Validation auc = 0.716901
Epoch 106
Validation auc = 0.716901
Epoch 107
Validation auc = 0.716432
Epoch 108
Validation auc = 0.714554
Epoch 109
Loss = 4.9650e-02, PNorm = 66.6316, GNorm = 0.8222, lr_0 = 8.4006e-04
Validation auc = 0.712676
Epoch 110
Validation auc = 0.715493
Epoch 111
Validation auc = 0.708920
Epoch 112
Validation auc = 0.706573
Epoch 113
Validation auc = 0.708451
Epoch 114
Loss = 3.1819e-02, PNorm = 66.7548, GNorm = 0.8702, lr_0 = 8.3288e-04
Validation auc = 0.708920
Epoch 115
Validation auc = 0.713615
Epoch 116
Validation auc = 0.714554
Epoch 117
Validation auc = 0.709859
Epoch 118
Validation auc = 0.707512
Epoch 119
Loss = 5.7063e-02, PNorm = 66.9072, GNorm = 0.4722, lr_0 = 8.2576e-04
Validation auc = 0.708451
Epoch 120
Validation auc = 0.708451
Epoch 121
Validation auc = 0.704225
Epoch 122
Validation auc = 0.704695
Epoch 123
Validation auc = 0.708451
Epoch 124
Loss = 6.8667e-02, PNorm = 67.0216, GNorm = 1.1741, lr_0 = 8.1870e-04
Validation auc = 0.712676
Epoch 125
Validation auc = 0.715493
Epoch 126
Validation auc = 0.711268
Epoch 127
Validation auc = 0.708451
Epoch 128
Validation auc = 0.704695
Epoch 129
Loss = 6.6235e-02, PNorm = 67.1534, GNorm = 0.6073, lr_0 = 8.1170e-04
Validation auc = 0.706103
Epoch 130
Validation auc = 0.700000
Epoch 131
Validation auc = 0.695775
Epoch 132
Validation auc = 0.691549
Epoch 133
Validation auc = 0.685915
Epoch 134
Loss = 2.5307e-02, PNorm = 67.2857, GNorm = 0.5079, lr_0 = 8.0476e-04
Validation auc = 0.681221
Epoch 135
Validation auc = 0.677465
Epoch 136
Validation auc = 0.687324
Epoch 137
Validation auc = 0.693897
Epoch 138
Validation auc = 0.696244
Epoch 139
Loss = 8.1165e-02, PNorm = 67.4140, GNorm = 1.4460, lr_0 = 7.9788e-04
Validation auc = 0.703756
Epoch 140
Validation auc = 0.700939
Epoch 141
Validation auc = 0.704225
Epoch 142
Validation auc = 0.710798
Epoch 143
Validation auc = 0.709859
Epoch 144
Loss = 5.3516e-02, PNorm = 67.5390, GNorm = 0.9193, lr_0 = 7.9106e-04
Validation auc = 0.708451
Epoch 145
Validation auc = 0.708451
Epoch 146
Validation auc = 0.700939
Epoch 147
Validation auc = 0.689671
Epoch 148
Validation auc = 0.682160
Epoch 149
Loss = 1.4739e-02, PNorm = 67.6774, GNorm = 0.2271, lr_0 = 7.8430e-04
Validation auc = 0.678873
Epoch 150
Validation auc = 0.681221
Epoch 151
Validation auc = 0.686385
Epoch 152
Validation auc = 0.699061
Epoch 153
Validation auc = 0.704225
Epoch 154
Loss = 3.3637e-02, PNorm = 67.7982, GNorm = 0.4150, lr_0 = 7.7759e-04
Validation auc = 0.707512
Epoch 155
Validation auc = 0.709859
Epoch 156
Validation auc = 0.709859
Epoch 157
Validation auc = 0.704225
Epoch 158
Validation auc = 0.694366
Epoch 159
Loss = 1.5501e-02, PNorm = 67.9168, GNorm = 0.3141, lr_0 = 7.7095e-04
Validation auc = 0.683568
Epoch 160
Validation auc = 0.689202
Epoch 161
Validation auc = 0.689202
Epoch 162
Validation auc = 0.691549
Epoch 163
Validation auc = 0.692958
Epoch 164
Loss = 1.1158e-02, PNorm = 68.0699, GNorm = 0.1293, lr_0 = 7.6436e-04
Validation auc = 0.693897
Epoch 165
Validation auc = 0.694366
Epoch 166
Validation auc = 0.699061
Epoch 167
Validation auc = 0.700939
Epoch 168
Validation auc = 0.700000
Epoch 169
Loss = 2.7118e-02, PNorm = 68.2165, GNorm = 0.4732, lr_0 = 7.5782e-04
Validation auc = 0.700469
Epoch 170
Validation auc = 0.697653
Epoch 171
Validation auc = 0.693427
Epoch 172
Validation auc = 0.692488
Epoch 173
Validation auc = 0.688732
Epoch 174
Loss = 1.5724e-02, PNorm = 68.3559, GNorm = 0.5351, lr_0 = 7.5134e-04
Validation auc = 0.692019
Epoch 175
Validation auc = 0.690610
Epoch 176
Validation auc = 0.689202
Epoch 177
Validation auc = 0.688263
Epoch 178
Validation auc = 0.687324
Epoch 179
Loss = 1.6692e-02, PNorm = 68.4367, GNorm = 0.5087, lr_0 = 7.4492e-04
Validation auc = 0.687324
Epoch 180
Validation auc = 0.685915
Epoch 181
Validation auc = 0.684038
Epoch 182
Validation auc = 0.682160
Epoch 183
Validation auc = 0.678873
Epoch 184
Loss = 4.7173e-03, PNorm = 68.5285, GNorm = 0.1248, lr_0 = 7.3855e-04
Validation auc = 0.673709
Epoch 185
Validation auc = 0.670423
Epoch 186
Validation auc = 0.669953
Epoch 187
Validation auc = 0.669953
Epoch 188
Validation auc = 0.670892
Epoch 189
Loss = 1.3759e-02, PNorm = 68.6029, GNorm = 0.6643, lr_0 = 7.3224e-04
Validation auc = 0.673239
Epoch 190
Validation auc = 0.680751
Epoch 191
Validation auc = 0.678873
Epoch 192
Validation auc = 0.677465
Epoch 193
Validation auc = 0.675117
Epoch 194
Loss = 5.2012e-03, PNorm = 68.6719, GNorm = 0.1438, lr_0 = 7.2598e-04
Validation auc = 0.676056
Epoch 195
Validation auc = 0.676995
Epoch 196
Validation auc = 0.682629
Epoch 197
Validation auc = 0.684507
Epoch 198
Validation auc = 0.687324
Epoch 199
Loss = 2.9142e-02, PNorm = 68.7251, GNorm = 1.5706, lr_0 = 7.1977e-04
Validation auc = 0.686385
Epoch 200
Validation auc = 0.688263
Epoch 201
Validation auc = 0.692958
Epoch 202
Validation auc = 0.695775
Epoch 203
Validation auc = 0.705164
Epoch 204
Loss = 2.6872e-02, PNorm = 68.7648, GNorm = 1.1013, lr_0 = 7.1362e-04
Validation auc = 0.711268
Epoch 205
Validation auc = 0.716432
Epoch 206
Validation auc = 0.719718
Epoch 207
Validation auc = 0.721127
Epoch 208
Validation auc = 0.719718
Epoch 209
Loss = 1.7864e-03, PNorm = 68.8210, GNorm = 0.0447, lr_0 = 7.0752e-04
Validation auc = 0.725352
Epoch 210
Validation auc = 0.726291
Epoch 211
Validation auc = 0.730516
Epoch 212
Validation auc = 0.726761
Epoch 213
Validation auc = 0.718779
Epoch 214
Loss = 3.5923e-03, PNorm = 68.9176, GNorm = 0.0451, lr_0 = 7.0147e-04
Validation auc = 0.713615
Epoch 215
Validation auc = 0.712207
Epoch 216
Validation auc = 0.715023
Epoch 217
Validation auc = 0.714554
Epoch 218
Validation auc = 0.711737
Epoch 219
Loss = 1.0264e-02, PNorm = 68.9759, GNorm = 0.1712, lr_0 = 6.9548e-04
Validation auc = 0.702817
Epoch 220
Validation auc = 0.697653
Epoch 221
Validation auc = 0.691080
Epoch 222
Validation auc = 0.689671
Epoch 223
Validation auc = 0.686854
Epoch 224
Loss = 3.5255e-02, PNorm = 69.0766, GNorm = 0.9807, lr_0 = 6.8953e-04
Validation auc = 0.683099
Epoch 225
Validation auc = 0.680282
Epoch 226
Validation auc = 0.680751
Epoch 227
Validation auc = 0.684977
Epoch 228
Validation auc = 0.691549
Epoch 229
Loss = 3.0458e-03, PNorm = 69.1616, GNorm = 0.0999, lr_0 = 6.8364e-04
Validation auc = 0.696244
Epoch 230
Validation auc = 0.700000
Epoch 231
Validation auc = 0.702347
Epoch 232
Validation auc = 0.701878
Epoch 233
Validation auc = 0.705634
Epoch 234
Loss = 5.0022e-03, PNorm = 69.2381, GNorm = 0.0478, lr_0 = 6.7779e-04
Validation auc = 0.704225
Epoch 235
Validation auc = 0.704695
Epoch 236
Validation auc = 0.705634
Epoch 237
Validation auc = 0.707512
Epoch 238
Validation auc = 0.704225
Epoch 239
Loss = 3.1446e-03, PNorm = 69.3112, GNorm = 0.0624, lr_0 = 6.7200e-04
Validation auc = 0.703286
Epoch 240
Validation auc = 0.704225
Epoch 241
Validation auc = 0.705164
Epoch 242
Validation auc = 0.717840
Epoch 243
Validation auc = 0.722066
Epoch 244
Loss = 1.1056e-02, PNorm = 69.3558, GNorm = 0.5509, lr_0 = 6.6625e-04
Validation auc = 0.717840
Epoch 245
Validation auc = 0.713615
Epoch 246
Validation auc = 0.710798
Epoch 247
Validation auc = 0.703756
Epoch 248
Validation auc = 0.706103
Epoch 249
Loss = 7.1197e-03, PNorm = 69.4331, GNorm = 0.4171, lr_0 = 6.6056e-04
Validation auc = 0.707042
Epoch 250
Validation auc = 0.701408
Epoch 251
Validation auc = 0.696714
Epoch 252
Validation auc = 0.699531
Epoch 253
Validation auc = 0.694366
Epoch 254
Loss = 1.6757e-02, PNorm = 69.5213, GNorm = 0.8452, lr_0 = 6.5491e-04
Validation auc = 0.694366
Epoch 255
Validation auc = 0.694366
Epoch 256
Validation auc = 0.694836
Epoch 257
Validation auc = 0.698122
Epoch 258
Validation auc = 0.696244
Epoch 259
Loss = 1.3566e-03, PNorm = 69.5812, GNorm = 0.0185, lr_0 = 6.4931e-04
Validation auc = 0.695775
Epoch 260
Validation auc = 0.695305
Epoch 261
Validation auc = 0.696714
Epoch 262
Validation auc = 0.693897
Epoch 263
Validation auc = 0.691080
Epoch 264
Loss = 1.3692e-02, PNorm = 69.6603, GNorm = 0.2139, lr_0 = 6.4376e-04
Validation auc = 0.684507
Epoch 265
Validation auc = 0.679343
Epoch 266
Validation auc = 0.673709
Epoch 267
Validation auc = 0.674648
Epoch 268
Validation auc = 0.686854
Epoch 269
Loss = 8.2804e-03, PNorm = 69.7315, GNorm = 0.1581, lr_0 = 6.3826e-04
Validation auc = 0.690141
Epoch 270
Validation auc = 0.694836
Epoch 271
Validation auc = 0.699061
Epoch 272
Validation auc = 0.698592
Epoch 273
Validation auc = 0.702817
Epoch 274
Loss = 1.0965e-02, PNorm = 69.8243, GNorm = 0.4018, lr_0 = 6.3280e-04
Validation auc = 0.709859
Epoch 275
Validation auc = 0.715962
Epoch 276
Validation auc = 0.725822
Epoch 277
Validation auc = 0.735211
Epoch 278
Validation auc = 0.739437
Epoch 279
Loss = 1.0965e-02, PNorm = 69.9093, GNorm = 0.1846, lr_0 = 6.2739e-04
Validation auc = 0.740845
Epoch 280
Validation auc = 0.737559
Epoch 281
Validation auc = 0.733333
Epoch 282
Validation auc = 0.727230
Epoch 283
Validation auc = 0.722535
Epoch 284
Loss = 2.6941e-03, PNorm = 69.9829, GNorm = 0.0601, lr_0 = 6.2203e-04
Validation auc = 0.721127
Epoch 285
Validation auc = 0.712207
Epoch 286
Validation auc = 0.707981
Epoch 287
Validation auc = 0.702817
Epoch 288
Validation auc = 0.701408
Epoch 289
Loss = 3.7103e-03, PNorm = 70.0235, GNorm = 0.1439, lr_0 = 6.1671e-04
Validation auc = 0.700939
Epoch 290
Validation auc = 0.702817
Epoch 291
Validation auc = 0.703286
Epoch 292
Validation auc = 0.702347
Epoch 293
Validation auc = 0.703756
Epoch 294
Loss = 3.7457e-03, PNorm = 70.0487, GNorm = 0.1364, lr_0 = 6.1144e-04
Validation auc = 0.709390
Epoch 295
Validation auc = 0.711737
Epoch 296
Validation auc = 0.715493
Epoch 297
Validation auc = 0.718310
Epoch 298
Validation auc = 0.719249
Epoch 299
Loss = 4.9502e-03, PNorm = 70.0803, GNorm = 0.1173, lr_0 = 6.0621e-04
Validation auc = 0.719718
Model 0 best validation auc = 0.768545 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.848886
Ensemble test auc = 0.848886
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.653052
Epoch 1
Validation auc = 0.725352
Epoch 2
Validation auc = 0.708920
Epoch 3
Validation auc = 0.745540
Epoch 4
Loss = 7.0606e-01, PNorm = 63.7726, GNorm = 2.7790, lr_0 = 6.5000e-04
Validation auc = 0.748357
Epoch 5
Validation auc = 0.744131
Epoch 6
Validation auc = 0.724413
Epoch 7
Validation auc = 0.722535
Epoch 8
Validation auc = 0.716901
Epoch 9
Loss = 4.4609e-01, PNorm = 63.9369, GNorm = 2.4002, lr_0 = 9.9743e-04
Validation auc = 0.728169
Epoch 10
Validation auc = 0.737559
Epoch 11
Validation auc = 0.736150
Epoch 12
Validation auc = 0.731925
Epoch 13
Validation auc = 0.726761
Epoch 14
Loss = 2.3656e-01, PNorm = 64.1191, GNorm = 1.9621, lr_0 = 9.8890e-04
Validation auc = 0.726761
Epoch 15
Validation auc = 0.732394
Epoch 16
Validation auc = 0.727700
Epoch 17
Validation auc = 0.713615
Epoch 18
Validation auc = 0.707512
Epoch 19
Loss = 2.1704e-01, PNorm = 64.2526, GNorm = 1.1354, lr_0 = 9.8045e-04
Validation auc = 0.702817
Epoch 20
Validation auc = 0.707981
Epoch 21
Validation auc = 0.708920
Epoch 22
Validation auc = 0.720188
Epoch 23
Validation auc = 0.729108
Epoch 24
Loss = 1.8440e-01, PNorm = 64.3564, GNorm = 1.4447, lr_0 = 9.7207e-04
Validation auc = 0.721127
Epoch 25
Validation auc = 0.718310
Epoch 26
Validation auc = 0.721127
Epoch 27
Validation auc = 0.718310
Epoch 28
Validation auc = 0.715493
Epoch 29
Loss = 2.3446e-01, PNorm = 64.4455, GNorm = 1.8427, lr_0 = 9.6376e-04
Validation auc = 0.708451
Epoch 30
Validation auc = 0.693897
Epoch 31
Validation auc = 0.690141
Epoch 32
Validation auc = 0.702347
Epoch 33
Validation auc = 0.714554
Epoch 34
Loss = 1.8982e-01, PNorm = 64.5378, GNorm = 1.2235, lr_0 = 9.5552e-04
Validation auc = 0.722066
Epoch 35
Validation auc = 0.726761
Epoch 36
Validation auc = 0.728169
Epoch 37
Validation auc = 0.724883
Epoch 38
Validation auc = 0.713615
Epoch 39
Loss = 1.2956e-01, PNorm = 64.6310, GNorm = 0.8814, lr_0 = 9.4735e-04
Validation auc = 0.709859
Epoch 40
Validation auc = 0.705634
Epoch 41
Validation auc = 0.711268
Epoch 42
Validation auc = 0.714554
Epoch 43
Validation auc = 0.715962
Epoch 44
Loss = 1.8520e-01, PNorm = 64.7282, GNorm = 1.3312, lr_0 = 9.3925e-04
Validation auc = 0.709859
Epoch 45
Validation auc = 0.704695
Epoch 46
Validation auc = 0.707981
Epoch 47
Validation auc = 0.709390
Epoch 48
Validation auc = 0.722066
Epoch 49
Loss = 7.5560e-02, PNorm = 64.8396, GNorm = 0.7490, lr_0 = 9.3122e-04
Validation auc = 0.725352
Epoch 50
Validation auc = 0.723474
Epoch 51
Validation auc = 0.712207
Epoch 52
Validation auc = 0.704225
Epoch 53
Validation auc = 0.715962
Epoch 54
Loss = 1.2458e-01, PNorm = 64.9512, GNorm = 1.1464, lr_0 = 9.2326e-04
Validation auc = 0.720188
Epoch 55
Validation auc = 0.717371
Epoch 56
Validation auc = 0.713615
Epoch 57
Validation auc = 0.713146
Epoch 58
Validation auc = 0.711268
Epoch 59
Loss = 1.6928e-01, PNorm = 65.0946, GNorm = 1.5438, lr_0 = 9.1537e-04
Validation auc = 0.709390
Epoch 60
Validation auc = 0.699061
Epoch 61
Validation auc = 0.696714
Epoch 62
Validation auc = 0.700000
Epoch 63
Validation auc = 0.698122
Epoch 64
Loss = 1.1463e-01, PNorm = 65.2326, GNorm = 1.0819, lr_0 = 9.0754e-04
Validation auc = 0.699531
Epoch 65
Validation auc = 0.710798
Epoch 66
Validation auc = 0.713615
Epoch 67
Validation auc = 0.715023
Epoch 68
Validation auc = 0.713146
Epoch 69
Loss = 8.0747e-02, PNorm = 65.3857, GNorm = 1.0829, lr_0 = 8.9978e-04
Validation auc = 0.722535
Epoch 70
Validation auc = 0.722535
Epoch 71
Validation auc = 0.722535
Epoch 72
Validation auc = 0.709390
Epoch 73
Validation auc = 0.707512
Epoch 74
Loss = 1.2634e-01, PNorm = 65.5442, GNorm = 0.7228, lr_0 = 8.9209e-04
Validation auc = 0.710798
Epoch 75
Validation auc = 0.708451
Epoch 76
Validation auc = 0.694366
Epoch 77
Validation auc = 0.700469
Epoch 78
Validation auc = 0.711737
Epoch 79
Loss = 8.7705e-02, PNorm = 65.7166, GNorm = 1.3981, lr_0 = 8.8447e-04
Validation auc = 0.701878
Epoch 80
Validation auc = 0.701408
Epoch 81
Validation auc = 0.703756
Epoch 82
Validation auc = 0.698122
Epoch 83
Validation auc = 0.697183
Epoch 84
Loss = 1.0553e-01, PNorm = 65.8810, GNorm = 1.0047, lr_0 = 8.7691e-04
Validation auc = 0.692958
Epoch 85
Validation auc = 0.699531
Epoch 86
Validation auc = 0.707512
Epoch 87
Validation auc = 0.711268
Epoch 88
Validation auc = 0.710798
Epoch 89
Loss = 6.8059e-02, PNorm = 66.0492, GNorm = 0.5107, lr_0 = 8.6941e-04
Validation auc = 0.708451
Epoch 90
Validation auc = 0.711737
Epoch 91
Validation auc = 0.710798
Epoch 92
Validation auc = 0.718779
Epoch 93
Validation auc = 0.722066
Epoch 94
Loss = 4.3603e-02, PNorm = 66.1943, GNorm = 0.5241, lr_0 = 8.6198e-04
Validation auc = 0.718779
Epoch 95
Validation auc = 0.709859
Epoch 96
Validation auc = 0.700469
Epoch 97
Validation auc = 0.702817
Epoch 98
Validation auc = 0.702817
Epoch 99
Loss = 7.3028e-02, PNorm = 66.3263, GNorm = 0.7405, lr_0 = 8.5461e-04
Validation auc = 0.699061
Epoch 100
Validation auc = 0.700469
Epoch 101
Validation auc = 0.693897
Epoch 102
Validation auc = 0.693427
Epoch 103
Validation auc = 0.716901
Epoch 104
Loss = 1.1513e-01, PNorm = 66.4450, GNorm = 3.8171, lr_0 = 8.4730e-04
Validation auc = 0.689671
Epoch 105
Validation auc = 0.676526
Epoch 106
Validation auc = 0.694836
Epoch 107
Validation auc = 0.716432
Epoch 108
Validation auc = 0.726761
Epoch 109
Loss = 7.6528e-02, PNorm = 66.6016, GNorm = 1.8251, lr_0 = 8.4006e-04
Validation auc = 0.722066
Epoch 110
Validation auc = 0.712207
Epoch 111
Validation auc = 0.708451
Epoch 112
Validation auc = 0.703756
Epoch 113
Validation auc = 0.699061
Epoch 114
Loss = 7.9007e-02, PNorm = 66.8189, GNorm = 0.6524, lr_0 = 8.3288e-04
Validation auc = 0.701408
Epoch 115
Validation auc = 0.706103
Epoch 116
Validation auc = 0.704695
Epoch 117
Validation auc = 0.700939
Epoch 118
Validation auc = 0.705164
Epoch 119
Loss = 3.1582e-02, PNorm = 66.9618, GNorm = 0.4602, lr_0 = 8.2576e-04
Validation auc = 0.713146
Epoch 120
Validation auc = 0.720188
Epoch 121
Validation auc = 0.730516
Epoch 122
Validation auc = 0.728169
Epoch 123
Validation auc = 0.728638
Epoch 124
Loss = 4.5117e-02, PNorm = 67.0672, GNorm = 1.0112, lr_0 = 8.1870e-04
Validation auc = 0.724883
Epoch 125
Validation auc = 0.722535
Epoch 126
Validation auc = 0.717840
Epoch 127
Validation auc = 0.707981
Epoch 128
Validation auc = 0.702817
Epoch 129
Loss = 3.8064e-02, PNorm = 67.1769, GNorm = 0.6557, lr_0 = 8.1170e-04
Validation auc = 0.696244
Epoch 130
Validation auc = 0.692958
Epoch 131
Validation auc = 0.697183
Epoch 132
Validation auc = 0.696714
Epoch 133
Validation auc = 0.692019
Epoch 134
Loss = 3.9998e-02, PNorm = 67.2849, GNorm = 0.5815, lr_0 = 8.0476e-04
Validation auc = 0.695775
Epoch 135
Validation auc = 0.692488
Epoch 136
Validation auc = 0.693427
Epoch 137
Validation auc = 0.690610
Epoch 138
Validation auc = 0.698592
Epoch 139
Loss = 5.1557e-02, PNorm = 67.3865, GNorm = 0.7662, lr_0 = 7.9788e-04
Validation auc = 0.702817
Epoch 140
Validation auc = 0.709859
Epoch 141
Validation auc = 0.706573
Epoch 142
Validation auc = 0.705164
Epoch 143
Validation auc = 0.705164
Epoch 144
Loss = 1.6398e-02, PNorm = 67.4997, GNorm = 0.1075, lr_0 = 7.9106e-04
Validation auc = 0.706103
Epoch 145
Validation auc = 0.711268
Epoch 146
Validation auc = 0.712207
Epoch 147
Validation auc = 0.719249
Epoch 148
Validation auc = 0.712676
Epoch 149
Loss = 4.3996e-02, PNorm = 67.5837, GNorm = 0.1631, lr_0 = 7.8430e-04
Validation auc = 0.704695
Epoch 150
Validation auc = 0.694366
Epoch 151
Validation auc = 0.692488
Epoch 152
Validation auc = 0.700939
Epoch 153
Validation auc = 0.702347
Epoch 154
Loss = 4.8579e-02, PNorm = 67.6623, GNorm = 0.7479, lr_0 = 7.7759e-04
Validation auc = 0.702347
Epoch 155
Validation auc = 0.703756
Epoch 156
Validation auc = 0.699061
Epoch 157
Validation auc = 0.699061
Epoch 158
Validation auc = 0.703286
Epoch 159
Loss = 3.3765e-02, PNorm = 67.7554, GNorm = 1.3921, lr_0 = 7.7095e-04
Validation auc = 0.706573
Epoch 160
Validation auc = 0.711737
Epoch 161
Validation auc = 0.707981
Epoch 162
Validation auc = 0.708920
Epoch 163
Validation auc = 0.705164
Epoch 164
Loss = 2.5953e-02, PNorm = 67.8609, GNorm = 0.4544, lr_0 = 7.6436e-04
Validation auc = 0.702817
Epoch 165
Validation auc = 0.696714
Epoch 166
Validation auc = 0.694366
Epoch 167
Validation auc = 0.685915
Epoch 168
Validation auc = 0.680751
Epoch 169
Loss = 3.1110e-02, PNorm = 67.9617, GNorm = 0.1420, lr_0 = 7.5782e-04
Validation auc = 0.684977
Epoch 170
Validation auc = 0.688263
Epoch 171
Validation auc = 0.689202
Epoch 172
Validation auc = 0.690141
Epoch 173
Validation auc = 0.689671
Epoch 174
Loss = 1.4776e-02, PNorm = 68.0590, GNorm = 0.6031, lr_0 = 7.5134e-04
Validation auc = 0.693427
Epoch 175
Validation auc = 0.696244
Epoch 176
Validation auc = 0.698122
Epoch 177
Validation auc = 0.699061
Epoch 178
Validation auc = 0.707042
Epoch 179
Loss = 2.2577e-02, PNorm = 68.1431, GNorm = 0.5646, lr_0 = 7.4492e-04
Validation auc = 0.709390
Epoch 180
Validation auc = 0.708451
Epoch 181
Validation auc = 0.706573
Epoch 182
Validation auc = 0.703756
Epoch 183
Validation auc = 0.704225
Epoch 184
Loss = 2.5446e-02, PNorm = 68.2419, GNorm = 0.1077, lr_0 = 7.3855e-04
Validation auc = 0.702347
Epoch 185
Validation auc = 0.699061
Epoch 186
Validation auc = 0.695305
Epoch 187
Validation auc = 0.694366
Epoch 188
Validation auc = 0.689202
Epoch 189
Loss = 2.1901e-02, PNorm = 68.3773, GNorm = 0.1738, lr_0 = 7.3224e-04
Validation auc = 0.689202
Epoch 190
Validation auc = 0.692019
Epoch 191
Validation auc = 0.696714
Epoch 192
Validation auc = 0.700000
Epoch 193
Validation auc = 0.705634
Epoch 194
Loss = 3.4452e-02, PNorm = 68.4935, GNorm = 0.3314, lr_0 = 7.2598e-04
Validation auc = 0.700939
Epoch 195
Validation auc = 0.691549
Epoch 196
Validation auc = 0.694836
Epoch 197
Validation auc = 0.698592
Epoch 198
Validation auc = 0.701408
Epoch 199
Loss = 3.7508e-02, PNorm = 68.6351, GNorm = 0.7976, lr_0 = 7.1977e-04
Validation auc = 0.700000
Epoch 200
Validation auc = 0.697653
Epoch 201
Validation auc = 0.691080
Epoch 202
Validation auc = 0.685446
Epoch 203
Validation auc = 0.683568
Epoch 204
Loss = 3.0995e-02, PNorm = 68.7794, GNorm = 0.8822, lr_0 = 7.1362e-04
Validation auc = 0.681221
Epoch 205
Validation auc = 0.674648
Epoch 206
Validation auc = 0.672300
Epoch 207
Validation auc = 0.668075
Epoch 208
Validation auc = 0.662911
Epoch 209
Loss = 5.0789e-02, PNorm = 68.8630, GNorm = 0.8142, lr_0 = 7.0752e-04
Validation auc = 0.674648
Epoch 210
Validation auc = 0.681690
Epoch 211
Validation auc = 0.686385
Epoch 212
Validation auc = 0.687324
Epoch 213
Validation auc = 0.689202
Epoch 214
Loss = 2.2138e-02, PNorm = 69.0038, GNorm = 0.6667, lr_0 = 7.0147e-04
Validation auc = 0.685446
Epoch 215
Validation auc = 0.687324
Epoch 216
Validation auc = 0.690141
Epoch 217
Validation auc = 0.697653
Epoch 218
Validation auc = 0.692488
Epoch 219
Loss = 9.4068e-03, PNorm = 69.1644, GNorm = 0.0382, lr_0 = 6.9548e-04
Validation auc = 0.690610
Epoch 220
Validation auc = 0.691549
Epoch 221
Validation auc = 0.696714
Epoch 222
Validation auc = 0.698122
Epoch 223
Validation auc = 0.702347
Epoch 224
Loss = 1.9811e-02, PNorm = 69.3094, GNorm = 0.6800, lr_0 = 6.8953e-04
Validation auc = 0.700939
Epoch 225
Validation auc = 0.702347
Epoch 226
Validation auc = 0.695775
Epoch 227
Validation auc = 0.694836
Epoch 228
Validation auc = 0.711737
Epoch 229
Loss = 1.1402e-02, PNorm = 69.4178, GNorm = 0.2566, lr_0 = 6.8364e-04
Validation auc = 0.722535
Epoch 230
Validation auc = 0.714085
Epoch 231
Validation auc = 0.710329
Epoch 232
Validation auc = 0.709859
Epoch 233
Validation auc = 0.707512
Epoch 234
Loss = 6.7864e-03, PNorm = 69.5107, GNorm = 0.1853, lr_0 = 6.7779e-04
Validation auc = 0.712207
Epoch 235
Validation auc = 0.713146
Epoch 236
Validation auc = 0.713146
Epoch 237
Validation auc = 0.710329
Epoch 238
Validation auc = 0.707512
Epoch 239
Loss = 5.6792e-03, PNorm = 69.5783, GNorm = 0.1542, lr_0 = 6.7200e-04
Validation auc = 0.704225
Epoch 240
Validation auc = 0.701408
Epoch 241
Validation auc = 0.699061
Epoch 242
Validation auc = 0.699061
Epoch 243
Validation auc = 0.699061
Epoch 244
Loss = 6.2983e-03, PNorm = 69.6543, GNorm = 0.0439, lr_0 = 6.6625e-04
Validation auc = 0.696714
Epoch 245
Validation auc = 0.692019
Epoch 246
Validation auc = 0.686854
Epoch 247
Validation auc = 0.683099
Epoch 248
Validation auc = 0.679812
Epoch 249
Loss = 3.9274e-03, PNorm = 69.7254, GNorm = 0.0507, lr_0 = 6.6056e-04
Validation auc = 0.677465
Epoch 250
Validation auc = 0.675117
Epoch 251
Validation auc = 0.672300
Epoch 252
Validation auc = 0.671362
Epoch 253
Validation auc = 0.668075
Epoch 254
Loss = 7.3248e-03, PNorm = 69.7961, GNorm = 0.2741, lr_0 = 6.5491e-04
Validation auc = 0.668075
Epoch 255
Validation auc = 0.664789
Epoch 256
Validation auc = 0.662441
Epoch 257
Validation auc = 0.662441
Epoch 258
Validation auc = 0.662911
Epoch 259
Loss = 5.0241e-03, PNorm = 69.8481, GNorm = 0.0882, lr_0 = 6.4931e-04
Validation auc = 0.664789
Epoch 260
Validation auc = 0.663380
Epoch 261
Validation auc = 0.663850
Epoch 262
Validation auc = 0.664789
Epoch 263
Validation auc = 0.665728
Epoch 264
Loss = 5.6002e-03, PNorm = 69.8876, GNorm = 0.3433, lr_0 = 6.4376e-04
Validation auc = 0.664789
Epoch 265
Validation auc = 0.666197
Epoch 266
Validation auc = 0.668545
Epoch 267
Validation auc = 0.671362
Epoch 268
Validation auc = 0.671831
Epoch 269
Loss = 1.9682e-02, PNorm = 69.9082, GNorm = 0.1521, lr_0 = 6.3826e-04
Validation auc = 0.672300
Epoch 270
Validation auc = 0.672770
Epoch 271
Validation auc = 0.672770
Epoch 272
Validation auc = 0.672300
Epoch 273
Validation auc = 0.672770
Epoch 274
Loss = 2.7729e-02, PNorm = 69.9627, GNorm = 0.8543, lr_0 = 6.3280e-04
Validation auc = 0.672770
Epoch 275
Validation auc = 0.674178
Epoch 276
Validation auc = 0.676526
Epoch 277
Validation auc = 0.677465
Epoch 278
Validation auc = 0.677934
Epoch 279
Loss = 1.2117e-02, PNorm = 70.0149, GNorm = 0.0976, lr_0 = 6.2739e-04
Validation auc = 0.675587
Epoch 280
Validation auc = 0.676995
Epoch 281
Validation auc = 0.675117
Epoch 282
Validation auc = 0.673709
Epoch 283
Validation auc = 0.673239
Epoch 284
Loss = 1.7438e-02, PNorm = 70.0693, GNorm = 0.0078, lr_0 = 6.2203e-04
Validation auc = 0.674178
Epoch 285
Validation auc = 0.676526
Epoch 286
Validation auc = 0.678404
Epoch 287
Validation auc = 0.678873
Epoch 288
Validation auc = 0.682629
Epoch 289
Loss = 1.1322e-02, PNorm = 70.0947, GNorm = 0.4071, lr_0 = 6.1671e-04
Validation auc = 0.682629
Epoch 290
Validation auc = 0.682629
Epoch 291
Validation auc = 0.683099
Epoch 292
Validation auc = 0.686854
Epoch 293
Validation auc = 0.688732
Epoch 294
Loss = 2.6212e-02, PNorm = 70.1332, GNorm = 0.1289, lr_0 = 6.1144e-04
Validation auc = 0.687793
Epoch 295
Validation auc = 0.685915
Epoch 296
Validation auc = 0.684507
Epoch 297
Validation auc = 0.682160
Epoch 298
Validation auc = 0.680751
Epoch 299
Loss = 1.7271e-03, PNorm = 70.1702, GNorm = 0.0367, lr_0 = 6.0621e-04
Validation auc = 0.678404
Model 0 best validation auc = 0.748357 on epoch 4
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.852498
Ensemble test auc = 0.852498
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.686385
Epoch 1
Validation auc = 0.712207
Epoch 2
Validation auc = 0.705164
Epoch 3
Validation auc = 0.738028
Epoch 4
Loss = 4.7387e-01, PNorm = 63.7736, GNorm = 1.6571, lr_0 = 6.5000e-04
Validation auc = 0.734272
Epoch 5
Validation auc = 0.741315
Epoch 6
Validation auc = 0.736620
Epoch 7
Validation auc = 0.737089
Epoch 8
Validation auc = 0.750704
Epoch 9
Loss = 3.8826e-01, PNorm = 63.9306, GNorm = 2.3490, lr_0 = 9.9743e-04
Validation auc = 0.745540
Epoch 10
Validation auc = 0.742254
Epoch 11
Validation auc = 0.743192
Epoch 12
Validation auc = 0.743662
Epoch 13
Validation auc = 0.739437
Epoch 14
Loss = 2.9738e-01, PNorm = 64.1015, GNorm = 0.9478, lr_0 = 9.8890e-04
Validation auc = 0.737559
Epoch 15
Validation auc = 0.740376
Epoch 16
Validation auc = 0.739906
Epoch 17
Validation auc = 0.735211
Epoch 18
Validation auc = 0.741315
Epoch 19
Loss = 2.2654e-01, PNorm = 64.2320, GNorm = 2.1908, lr_0 = 9.8045e-04
Validation auc = 0.744601
Epoch 20
Validation auc = 0.750235
Epoch 21
Validation auc = 0.744131
Epoch 22
Validation auc = 0.736150
Epoch 23
Validation auc = 0.732394
Epoch 24
Loss = 1.7918e-01, PNorm = 64.3357, GNorm = 0.9766, lr_0 = 9.7207e-04
Validation auc = 0.726291
Epoch 25
Validation auc = 0.722535
Epoch 26
Validation auc = 0.713615
Epoch 27
Validation auc = 0.713615
Epoch 28
Validation auc = 0.720657
Epoch 29
Loss = 2.2428e-01, PNorm = 64.4224, GNorm = 1.2435, lr_0 = 9.6376e-04
Validation auc = 0.718779
Epoch 30
Validation auc = 0.715493
Epoch 31
Validation auc = 0.703756
Epoch 32
Validation auc = 0.705634
Epoch 33
Validation auc = 0.720188
Epoch 34
Loss = 2.2007e-01, PNorm = 64.5146, GNorm = 1.7545, lr_0 = 9.5552e-04
Validation auc = 0.722535
Epoch 35
Validation auc = 0.719249
Epoch 36
Validation auc = 0.716432
Epoch 37
Validation auc = 0.727230
Epoch 38
Validation auc = 0.735681
Epoch 39
Loss = 1.8344e-01, PNorm = 64.6154, GNorm = 2.7300, lr_0 = 9.4735e-04
Validation auc = 0.738028
Epoch 40
Validation auc = 0.733333
Epoch 41
Validation auc = 0.721596
Epoch 42
Validation auc = 0.715493
Epoch 43
Validation auc = 0.725352
Epoch 44
Loss = 1.2787e-01, PNorm = 64.7130, GNorm = 1.4319, lr_0 = 9.3925e-04
Validation auc = 0.732394
Epoch 45
Validation auc = 0.729577
Epoch 46
Validation auc = 0.714554
Epoch 47
Validation auc = 0.711737
Epoch 48
Validation auc = 0.719718
Epoch 49
Loss = 1.0167e-01, PNorm = 64.8306, GNorm = 0.5675, lr_0 = 9.3122e-04
Validation auc = 0.723944
Epoch 50
Validation auc = 0.717840
Epoch 51
Validation auc = 0.707042
Epoch 52
Validation auc = 0.706573
Epoch 53
Validation auc = 0.706103
Epoch 54
Loss = 7.2996e-02, PNorm = 64.9541, GNorm = 0.8734, lr_0 = 9.2326e-04
Validation auc = 0.707512
Epoch 55
Validation auc = 0.714085
Epoch 56
Validation auc = 0.720188
Epoch 57
Validation auc = 0.722535
Epoch 58
Validation auc = 0.725352
Epoch 59
Loss = 1.8836e-01, PNorm = 65.0817, GNorm = 1.2464, lr_0 = 9.1537e-04
Validation auc = 0.735211
Epoch 60
Validation auc = 0.743192
Epoch 61
Validation auc = 0.741315
Epoch 62
Validation auc = 0.738028
Epoch 63
Validation auc = 0.737559
Epoch 64
Loss = 8.4839e-02, PNorm = 65.2156, GNorm = 0.4802, lr_0 = 9.0754e-04
Validation auc = 0.733803
Epoch 65
Validation auc = 0.732394
Epoch 66
Validation auc = 0.724883
Epoch 67
Validation auc = 0.720188
Epoch 68
Validation auc = 0.716901
Epoch 69
Loss = 1.4861e-01, PNorm = 65.3554, GNorm = 1.4848, lr_0 = 8.9978e-04
Validation auc = 0.717840
Epoch 70
Validation auc = 0.715962
Epoch 71
Validation auc = 0.707042
Epoch 72
Validation auc = 0.714085
Epoch 73
Validation auc = 0.723474
Epoch 74
Loss = 6.2703e-02, PNorm = 65.4867, GNorm = 0.6818, lr_0 = 8.9209e-04
Validation auc = 0.727700
Epoch 75
Validation auc = 0.725352
Epoch 76
Validation auc = 0.721596
Epoch 77
Validation auc = 0.715962
Epoch 78
Validation auc = 0.720188
Epoch 79
Loss = 7.3514e-02, PNorm = 65.6238, GNorm = 1.0412, lr_0 = 8.8447e-04
Validation auc = 0.722535
Epoch 80
Validation auc = 0.717840
Epoch 81
Validation auc = 0.710329
Epoch 82
Validation auc = 0.706573
Epoch 83
Validation auc = 0.709390
Epoch 84
Loss = 9.7768e-02, PNorm = 65.7451, GNorm = 0.7829, lr_0 = 8.7691e-04
Validation auc = 0.721127
Epoch 85
Validation auc = 0.736620
Epoch 86
Validation auc = 0.729108
Epoch 87
Validation auc = 0.728169
Epoch 88
Validation auc = 0.734272
Epoch 89
Loss = 1.1847e-01, PNorm = 65.9157, GNorm = 1.7323, lr_0 = 8.6941e-04
Validation auc = 0.730516
Epoch 90
Validation auc = 0.722066
Epoch 91
Validation auc = 0.714554
Epoch 92
Validation auc = 0.714085
Epoch 93
Validation auc = 0.707981
Epoch 94
Loss = 1.0202e-01, PNorm = 66.0507, GNorm = 1.3139, lr_0 = 8.6198e-04
Validation auc = 0.718310
Epoch 95
Validation auc = 0.720188
Epoch 96
Validation auc = 0.719249
Epoch 97
Validation auc = 0.720188
Epoch 98
Validation auc = 0.724413
Epoch 99
Loss = 5.8026e-02, PNorm = 66.2030, GNorm = 0.8852, lr_0 = 8.5461e-04
Validation auc = 0.724413
Epoch 100
Validation auc = 0.728169
Epoch 101
Validation auc = 0.730986
Epoch 102
Validation auc = 0.726291
Epoch 103
Validation auc = 0.720188
Epoch 104
Loss = 5.4016e-02, PNorm = 66.3737, GNorm = 0.7060, lr_0 = 8.4730e-04
Validation auc = 0.709390
Epoch 105
Validation auc = 0.711737
Epoch 106
Validation auc = 0.712676
Epoch 107
Validation auc = 0.719249
Epoch 108
Validation auc = 0.727230
Epoch 109
Loss = 1.3182e-01, PNorm = 66.5800, GNorm = 1.0651, lr_0 = 8.4006e-04
Validation auc = 0.728169
Epoch 110
Validation auc = 0.733333
Epoch 111
Validation auc = 0.733333
Epoch 112
Validation auc = 0.720657
Epoch 113
Validation auc = 0.719249
Epoch 114
Loss = 7.9943e-02, PNorm = 66.7812, GNorm = 0.3204, lr_0 = 8.3288e-04
Validation auc = 0.725352
Epoch 115
Validation auc = 0.723944
Epoch 116
Validation auc = 0.722535
Epoch 117
Validation auc = 0.719718
Epoch 118
Validation auc = 0.717840
Epoch 119
Loss = 6.6541e-02, PNorm = 66.9368, GNorm = 0.6870, lr_0 = 8.2576e-04
Validation auc = 0.712207
Epoch 120
Validation auc = 0.713615
Epoch 121
Validation auc = 0.707981
Epoch 122
Validation auc = 0.707981
Epoch 123
Validation auc = 0.705634
Epoch 124
Loss = 5.2089e-02, PNorm = 67.0945, GNorm = 0.5936, lr_0 = 8.1870e-04
Validation auc = 0.701408
Epoch 125
Validation auc = 0.692488
Epoch 126
Validation auc = 0.689202
Epoch 127
Validation auc = 0.684038
Epoch 128
Validation auc = 0.676526
Epoch 129
Loss = 1.0885e-02, PNorm = 67.2267, GNorm = 0.1519, lr_0 = 8.1170e-04
Validation auc = 0.673709
Epoch 130
Validation auc = 0.670892
Epoch 131
Validation auc = 0.670423
Epoch 132
Validation auc = 0.670423
Epoch 133
Validation auc = 0.672770
Epoch 134
Loss = 3.3258e-02, PNorm = 67.3597, GNorm = 0.4427, lr_0 = 8.0476e-04
Validation auc = 0.676995
Epoch 135
Validation auc = 0.674648
Epoch 136
Validation auc = 0.676526
Epoch 137
Validation auc = 0.677934
Epoch 138
Validation auc = 0.679343
Epoch 139
Loss = 1.0620e-02, PNorm = 67.4776, GNorm = 0.1692, lr_0 = 7.9788e-04
Validation auc = 0.680751
Epoch 140
Validation auc = 0.687793
Epoch 141
Validation auc = 0.690610
Epoch 142
Validation auc = 0.691549
Epoch 143
Validation auc = 0.693427
Epoch 144
Loss = 1.4647e-02, PNorm = 67.5936, GNorm = 0.3220, lr_0 = 7.9106e-04
Validation auc = 0.694366
Epoch 145
Validation auc = 0.694366
Epoch 146
Validation auc = 0.700469
Epoch 147
Validation auc = 0.715023
Epoch 148
Validation auc = 0.699531
Epoch 149
Loss = 4.1118e-02, PNorm = 67.6973, GNorm = 0.4722, lr_0 = 7.8430e-04
Validation auc = 0.694366
Epoch 150
Validation auc = 0.688732
Epoch 151
Validation auc = 0.681690
Epoch 152
Validation auc = 0.677465
Epoch 153
Validation auc = 0.675587
Epoch 154
Loss = 2.8446e-02, PNorm = 67.8187, GNorm = 0.6803, lr_0 = 7.7759e-04
Validation auc = 0.672770
Epoch 155
Validation auc = 0.669484
Epoch 156
Validation auc = 0.666197
Epoch 157
Validation auc = 0.674648
Epoch 158
Validation auc = 0.683568
Epoch 159
Loss = 1.7046e-02, PNorm = 67.9193, GNorm = 0.2455, lr_0 = 7.7095e-04
Validation auc = 0.691549
Epoch 160
Validation auc = 0.695775
Epoch 161
Validation auc = 0.698122
Epoch 162
Validation auc = 0.697183
Epoch 163
Validation auc = 0.698592
Epoch 164
Loss = 7.0848e-02, PNorm = 68.0096, GNorm = 1.3561, lr_0 = 7.6436e-04
Validation auc = 0.693897
Epoch 165
Validation auc = 0.701408
Epoch 166
Validation auc = 0.701878
Epoch 167
Validation auc = 0.701408
Epoch 168
Validation auc = 0.701408
Epoch 169
Loss = 4.3206e-02, PNorm = 68.1215, GNorm = 0.3234, lr_0 = 7.5782e-04
Validation auc = 0.694366
Epoch 170
Validation auc = 0.696244
Epoch 171
Validation auc = 0.699531
Epoch 172
Validation auc = 0.700000
Epoch 173
Validation auc = 0.702817
Epoch 174
Loss = 2.2769e-02, PNorm = 68.2902, GNorm = 0.2075, lr_0 = 7.5134e-04
Validation auc = 0.699531
Epoch 175
Validation auc = 0.699061
Epoch 176
Validation auc = 0.697183
Epoch 177
Validation auc = 0.696244
Epoch 178
Validation auc = 0.694366
Epoch 179
Loss = 2.2843e-02, PNorm = 68.4106, GNorm = 0.1957, lr_0 = 7.4492e-04
Validation auc = 0.694836
Epoch 180
Validation auc = 0.691549
Epoch 181
Validation auc = 0.696714
Epoch 182
Validation auc = 0.706573
Epoch 183
Validation auc = 0.708451
Epoch 184
Loss = 1.2010e-02, PNorm = 68.5253, GNorm = 0.4235, lr_0 = 7.3855e-04
Validation auc = 0.706103
Epoch 185
Validation auc = 0.708451
Epoch 186
Validation auc = 0.707981
Epoch 187
Validation auc = 0.706573
Epoch 188
Validation auc = 0.710798
Epoch 189
Loss = 1.0022e-02, PNorm = 68.6317, GNorm = 0.1901, lr_0 = 7.3224e-04
Validation auc = 0.709859
Epoch 190
Validation auc = 0.707512
Epoch 191
Validation auc = 0.701878
Epoch 192
Validation auc = 0.692019
Epoch 193
Validation auc = 0.689202
Epoch 194
Loss = 5.7311e-03, PNorm = 68.7384, GNorm = 0.1373, lr_0 = 7.2598e-04
Validation auc = 0.692019
Epoch 195
Validation auc = 0.691080
Epoch 196
Validation auc = 0.692488
Epoch 197
Validation auc = 0.701408
Epoch 198
Validation auc = 0.713615
Epoch 199
Loss = 3.4534e-02, PNorm = 68.8309, GNorm = 0.4752, lr_0 = 7.1977e-04
Validation auc = 0.725352
Epoch 200
Validation auc = 0.735681
Epoch 201
Validation auc = 0.740845
Epoch 202
Validation auc = 0.741315
Epoch 203
Validation auc = 0.744131
Epoch 204
Loss = 2.1901e-02, PNorm = 68.9431, GNorm = 0.9012, lr_0 = 7.1362e-04
Validation auc = 0.744131
Epoch 205
Validation auc = 0.761972
Epoch 206
Validation auc = 0.767136
Epoch 207
Validation auc = 0.752582
Epoch 208
Validation auc = 0.738498
Epoch 209
Loss = 1.1994e-02, PNorm = 69.0611, GNorm = 0.1851, lr_0 = 7.0752e-04
Validation auc = 0.726291
Epoch 210
Validation auc = 0.721127
Epoch 211
Validation auc = 0.715493
Epoch 212
Validation auc = 0.706103
Epoch 213
Validation auc = 0.704695
Epoch 214
Loss = 1.6756e-02, PNorm = 69.1671, GNorm = 0.0999, lr_0 = 7.0147e-04
Validation auc = 0.706573
Epoch 215
Validation auc = 0.708920
Epoch 216
Validation auc = 0.706103
Epoch 217
Validation auc = 0.704225
Epoch 218
Validation auc = 0.702817
Epoch 219
Loss = 1.4837e-02, PNorm = 69.2594, GNorm = 0.1485, lr_0 = 6.9548e-04
Validation auc = 0.703756
Epoch 220
Validation auc = 0.707042
Epoch 221
Validation auc = 0.709859
Epoch 222
Validation auc = 0.709859
Epoch 223
Validation auc = 0.709390
Epoch 224
Loss = 5.0231e-03, PNorm = 69.3611, GNorm = 0.1159, lr_0 = 6.8953e-04
Validation auc = 0.706573
Epoch 225
Validation auc = 0.711268
Epoch 226
Validation auc = 0.711737
Epoch 227
Validation auc = 0.715493
Epoch 228
Validation auc = 0.717371
Epoch 229
Loss = 9.8026e-03, PNorm = 69.4425, GNorm = 0.1312, lr_0 = 6.8364e-04
Validation auc = 0.718310
Epoch 230
Validation auc = 0.718310
Epoch 231
Validation auc = 0.717840
Epoch 232
Validation auc = 0.717840
Epoch 233
Validation auc = 0.719249
Epoch 234
Loss = 2.4950e-03, PNorm = 69.5069, GNorm = 0.0525, lr_0 = 6.7779e-04
Validation auc = 0.721127
Epoch 235
Validation auc = 0.721596
Epoch 236
Validation auc = 0.720188
Epoch 237
Validation auc = 0.722535
Epoch 238
Validation auc = 0.722066
Epoch 239
Loss = 7.1389e-03, PNorm = 69.5636, GNorm = 0.2836, lr_0 = 6.7200e-04
Validation auc = 0.720657
Epoch 240
Validation auc = 0.716901
Epoch 241
Validation auc = 0.710329
Epoch 242
Validation auc = 0.709390
Epoch 243
Validation auc = 0.709390
Epoch 244
Loss = 2.6096e-02, PNorm = 69.6095, GNorm = 0.1634, lr_0 = 6.6625e-04
Validation auc = 0.707512
Epoch 245
Validation auc = 0.708920
Epoch 246
Validation auc = 0.710798
Epoch 247
Validation auc = 0.708451
Epoch 248
Validation auc = 0.706573
Epoch 249
Loss = 7.3187e-03, PNorm = 69.6379, GNorm = 0.3478, lr_0 = 6.6056e-04
Validation auc = 0.706573
Epoch 250
Validation auc = 0.705164
Epoch 251
Validation auc = 0.702817
Epoch 252
Validation auc = 0.702817
Epoch 253
Validation auc = 0.700939
Epoch 254
Loss = 4.1751e-03, PNorm = 69.6689, GNorm = 0.0822, lr_0 = 6.5491e-04
Validation auc = 0.702347
Epoch 255
Validation auc = 0.701408
Epoch 256
Validation auc = 0.701408
Epoch 257
Validation auc = 0.703286
Epoch 258
Validation auc = 0.704695
Epoch 259
Loss = 1.7412e-03, PNorm = 69.6985, GNorm = 0.0447, lr_0 = 6.4931e-04
Validation auc = 0.705634
Epoch 260
Validation auc = 0.705634
Epoch 261
Validation auc = 0.707042
Epoch 262
Validation auc = 0.718310
Epoch 263
Validation auc = 0.723944
Epoch 264
Loss = 4.7252e-03, PNorm = 69.7290, GNorm = 0.0911, lr_0 = 6.4376e-04
Validation auc = 0.733333
Epoch 265
Validation auc = 0.739906
Epoch 266
Validation auc = 0.747887
Epoch 267
Validation auc = 0.749765
Epoch 268
Validation auc = 0.749296
Epoch 269
Loss = 3.8075e-03, PNorm = 69.7877, GNorm = 0.1154, lr_0 = 6.3826e-04
Validation auc = 0.749765
Epoch 270
Validation auc = 0.750704
Epoch 271
Validation auc = 0.752113
Epoch 272
Validation auc = 0.749765
Epoch 273
Validation auc = 0.747887
Epoch 274
Loss = 3.1640e-03, PNorm = 69.8368, GNorm = 0.0170, lr_0 = 6.3280e-04
Validation auc = 0.746479
Epoch 275
Validation auc = 0.746948
Epoch 276
Validation auc = 0.746479
Epoch 277
Validation auc = 0.744131
Epoch 278
Validation auc = 0.746479
Epoch 279
Loss = 4.3414e-03, PNorm = 69.8774, GNorm = 0.1570, lr_0 = 6.2739e-04
Validation auc = 0.747418
Epoch 280
Validation auc = 0.746948
Epoch 281
Validation auc = 0.748826
Epoch 282
Validation auc = 0.747418
Epoch 283
Validation auc = 0.740376
Epoch 284
Loss = 1.2601e-02, PNorm = 69.9386, GNorm = 0.1054, lr_0 = 6.2203e-04
Validation auc = 0.730986
Epoch 285
Validation auc = 0.721596
Epoch 286
Validation auc = 0.713615
Epoch 287
Validation auc = 0.702817
Epoch 288
Validation auc = 0.698592
Epoch 289
Loss = 4.3404e-03, PNorm = 70.0240, GNorm = 0.0405, lr_0 = 6.1671e-04
Validation auc = 0.695305
Epoch 290
Validation auc = 0.693897
Epoch 291
Validation auc = 0.695775
Epoch 292
Validation auc = 0.695775
Epoch 293
Validation auc = 0.695775
Epoch 294
Loss = 1.7516e-03, PNorm = 70.0795, GNorm = 0.0653, lr_0 = 6.1144e-04
Validation auc = 0.698122
Epoch 295
Validation auc = 0.702347
Epoch 296
Validation auc = 0.705164
Epoch 297
Validation auc = 0.707042
Epoch 298
Validation auc = 0.708920
Epoch 299
Loss = 1.8076e-03, PNorm = 70.1102, GNorm = 0.0777, lr_0 = 6.0621e-04
Validation auc = 0.710329
Model 0 best validation auc = 0.767136 on epoch 206
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.889223
Ensemble test auc = 0.889223
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1200, bias=False)
        (W_h): Linear(in_features=1200, out_features=1200, bias=False)
        (W_o): Linear(in_features=1333, out_features=1200, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=1500, out_features=1200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=1200, out_features=1, bias=True)
  )
)
Number of parameters = 5,019,601
Moving model to cuda
Epoch 0
Validation auc = 0.634272
Epoch 1
Validation auc = 0.714085
Epoch 2
Validation auc = 0.717840
Epoch 3
Validation auc = 0.729577
Epoch 4
Loss = 5.4865e-01, PNorm = 63.7700, GNorm = 2.5870, lr_0 = 6.5000e-04
Validation auc = 0.748826
Epoch 5
Validation auc = 0.746009
Epoch 6
Validation auc = 0.755399
Epoch 7
Validation auc = 0.758216
Epoch 8
Validation auc = 0.748826
Epoch 9
Loss = 3.7236e-01, PNorm = 63.9287, GNorm = 2.7771, lr_0 = 9.9743e-04
Validation auc = 0.745070
Epoch 10
Validation auc = 0.752582
Epoch 11
Validation auc = 0.750235
Epoch 12
Validation auc = 0.738967
Epoch 13
Validation auc = 0.738967
Epoch 14
Loss = 3.0283e-01, PNorm = 64.1123, GNorm = 1.3841, lr_0 = 9.8890e-04
Validation auc = 0.739437
Epoch 15
Validation auc = 0.723944
Epoch 16
Validation auc = 0.707042
Epoch 17
Validation auc = 0.692019
Epoch 18
Validation auc = 0.690141
Epoch 19
Loss = 2.5682e-01, PNorm = 64.2581, GNorm = 1.3259, lr_0 = 9.8045e-04
Validation auc = 0.699531
Epoch 20
Validation auc = 0.720657
Epoch 21
Validation auc = 0.723005
Epoch 22
Validation auc = 0.741784
Epoch 23
Validation auc = 0.746948
Epoch 24
Loss = 2.5932e-01, PNorm = 64.3697, GNorm = 3.4960, lr_0 = 9.7207e-04
Validation auc = 0.742723
Epoch 25
Validation auc = 0.735681
Epoch 26
Validation auc = 0.736150
Epoch 27
Validation auc = 0.747418
Epoch 28
Validation auc = 0.746479
Epoch 29
Loss = 2.2104e-01, PNorm = 64.4737, GNorm = 1.6504, lr_0 = 9.6376e-04
Validation auc = 0.739906
Epoch 30
Validation auc = 0.727700
Epoch 31
Validation auc = 0.729108
Epoch 32
Validation auc = 0.733803
Epoch 33
Validation auc = 0.731455
Epoch 34
Loss = 1.4808e-01, PNorm = 64.5657, GNorm = 1.0681, lr_0 = 9.5552e-04
Validation auc = 0.724883
Epoch 35
Validation auc = 0.720657
Epoch 36
Validation auc = 0.724883
Epoch 37
Validation auc = 0.734742
Epoch 38
Validation auc = 0.737089
Epoch 39
Loss = 2.5817e-01, PNorm = 64.6589, GNorm = 0.9547, lr_0 = 9.4735e-04
Validation auc = 0.731455
Epoch 40
Validation auc = 0.730516
Epoch 41
Validation auc = 0.727700
Epoch 42
Validation auc = 0.727700
Epoch 43
Validation auc = 0.724413
Epoch 44
Loss = 2.7485e-01, PNorm = 64.7709, GNorm = 1.6311, lr_0 = 9.3925e-04
Validation auc = 0.714085
Epoch 45
Validation auc = 0.706103
Epoch 46
Validation auc = 0.696244
Epoch 47
Validation auc = 0.696244
Epoch 48
Validation auc = 0.707981
Epoch 49
Loss = 1.4309e-01, PNorm = 64.8973, GNorm = 0.9842, lr_0 = 9.3122e-04
Validation auc = 0.716432
Epoch 50
Validation auc = 0.714085
Epoch 51
Validation auc = 0.714554
Epoch 52
Validation auc = 0.715023
Epoch 53
Validation auc = 0.721596
Epoch 54
Loss = 1.4378e-01, PNorm = 65.0215, GNorm = 1.9917, lr_0 = 9.2326e-04
Validation auc = 0.720188
Epoch 55
Validation auc = 0.717371
Epoch 56
Validation auc = 0.713146
Epoch 57
Validation auc = 0.720188
Epoch 58
Validation auc = 0.718310
Epoch 59
Loss = 1.2521e-01, PNorm = 65.1547, GNorm = 1.6528, lr_0 = 9.1537e-04
Validation auc = 0.721596
Epoch 60
Validation auc = 0.715962
Epoch 61
Validation auc = 0.720657
Epoch 62
Validation auc = 0.720188
Epoch 63
Validation auc = 0.715493
Epoch 64
Loss = 1.0682e-01, PNorm = 65.3043, GNorm = 0.6530, lr_0 = 9.0754e-04
Validation auc = 0.715493
Epoch 65
Validation auc = 0.713615
Epoch 66
Validation auc = 0.711737
Epoch 67
Validation auc = 0.709859
Epoch 68
Validation auc = 0.708451
Epoch 69
Loss = 6.2846e-02, PNorm = 65.4322, GNorm = 0.5744, lr_0 = 8.9978e-04
Validation auc = 0.711268
Epoch 70
Validation auc = 0.715493
Epoch 71
Validation auc = 0.717840
Epoch 72
Validation auc = 0.715493
Epoch 73
Validation auc = 0.708920
Epoch 74
Loss = 8.8389e-02, PNorm = 65.5480, GNorm = 1.6707, lr_0 = 8.9209e-04
Validation auc = 0.713615
Epoch 75
Validation auc = 0.715023
Epoch 76
Validation auc = 0.707512
Epoch 77
Validation auc = 0.702347
Epoch 78
Validation auc = 0.698122
Epoch 79
Loss = 9.3533e-02, PNorm = 65.6943, GNorm = 1.1188, lr_0 = 8.8447e-04
Validation auc = 0.700469
Epoch 80
Validation auc = 0.695775
Epoch 81
Validation auc = 0.693897
Epoch 82
Validation auc = 0.690141
Epoch 83
Validation auc = 0.688732
Epoch 84
Loss = 4.9031e-02, PNorm = 65.8393, GNorm = 1.0767, lr_0 = 8.7691e-04
Validation auc = 0.690141
Epoch 85
Validation auc = 0.695775
Epoch 86
Validation auc = 0.702817
Epoch 87
Validation auc = 0.704695
Epoch 88
Validation auc = 0.706573
Epoch 89
Loss = 9.1502e-02, PNorm = 65.9742, GNorm = 1.3587, lr_0 = 8.6941e-04
Validation auc = 0.709390
Epoch 90
Validation auc = 0.715962
Epoch 91
Validation auc = 0.716432
Epoch 92
Validation auc = 0.716901
Epoch 93
Validation auc = 0.721127
Epoch 94
Loss = 5.6090e-02, PNorm = 66.1342, GNorm = 0.4613, lr_0 = 8.6198e-04
Validation auc = 0.724413
Epoch 95
Validation auc = 0.721596
Epoch 96
Validation auc = 0.721127
Epoch 97
Validation auc = 0.704695
Epoch 98
Validation auc = 0.703756
Epoch 99
Loss = 1.0112e-01, PNorm = 66.2929, GNorm = 1.0937, lr_0 = 8.5461e-04
Validation auc = 0.708920
Epoch 100
Validation auc = 0.709859
Epoch 101
Validation auc = 0.717371
Epoch 102
Validation auc = 0.709390
Epoch 103
Validation auc = 0.711737
Epoch 104
Loss = 3.2770e-02, PNorm = 66.4634, GNorm = 0.6603, lr_0 = 8.4730e-04
Validation auc = 0.715493
Epoch 105
Validation auc = 0.711268
Epoch 106
Validation auc = 0.711737
Epoch 107
Validation auc = 0.707981
Epoch 108
Validation auc = 0.700939
Epoch 109
Loss = 4.8404e-02, PNorm = 66.6062, GNorm = 0.7931, lr_0 = 8.4006e-04
Validation auc = 0.696244
Epoch 110
Validation auc = 0.697653
Epoch 111
Validation auc = 0.696244
Epoch 112
Validation auc = 0.695305
Epoch 113
Validation auc = 0.695775
Epoch 114
Loss = 2.8919e-02, PNorm = 66.7421, GNorm = 0.5712, lr_0 = 8.3288e-04
Validation auc = 0.698122
Epoch 115
Validation auc = 0.711737
Epoch 116
Validation auc = 0.711737
Epoch 117
Validation auc = 0.703286
Epoch 118
Validation auc = 0.692488
Epoch 119
Loss = 3.1668e-02, PNorm = 66.8713, GNorm = 0.5809, lr_0 = 8.2576e-04
Validation auc = 0.689671
Epoch 120
Validation auc = 0.684977
Epoch 121
Validation auc = 0.681221
Epoch 122
Validation auc = 0.677934
Epoch 123
Validation auc = 0.671362
Epoch 124
Loss = 4.7163e-02, PNorm = 67.0163, GNorm = 0.7899, lr_0 = 8.1870e-04
Validation auc = 0.667136
Epoch 125
Validation auc = 0.672770
Epoch 126
Validation auc = 0.680751
Epoch 127
Validation auc = 0.680751
Epoch 128
Validation auc = 0.682629
Epoch 129
Loss = 1.7257e-02, PNorm = 67.1276, GNorm = 0.5193, lr_0 = 8.1170e-04
Validation auc = 0.685915
Epoch 130
Validation auc = 0.691080
Epoch 131
Validation auc = 0.693427
Epoch 132
Validation auc = 0.698592
Epoch 133
Validation auc = 0.698122
Epoch 134
Loss = 2.7448e-02, PNorm = 67.2507, GNorm = 0.5549, lr_0 = 8.0476e-04
Validation auc = 0.696244
Epoch 135
Validation auc = 0.693427
Epoch 136
Validation auc = 0.691549
Epoch 137
Validation auc = 0.695305
Epoch 138
Validation auc = 0.697653
Epoch 139
Loss = 4.8571e-02, PNorm = 67.3708, GNorm = 0.9903, lr_0 = 7.9788e-04
Validation auc = 0.695305
Epoch 140
Validation auc = 0.695305
Epoch 141
Validation auc = 0.697183
Epoch 142
Validation auc = 0.696244
Epoch 143
Validation auc = 0.688732
Epoch 144
Loss = 5.5893e-02, PNorm = 67.4988, GNorm = 0.7349, lr_0 = 7.9106e-04
Validation auc = 0.691549
Epoch 145
Validation auc = 0.689202
Epoch 146
Validation auc = 0.694366
Epoch 147
Validation auc = 0.700000
Epoch 148
Validation auc = 0.702817
Epoch 149
Loss = 3.1257e-02, PNorm = 67.6399, GNorm = 0.3281, lr_0 = 7.8430e-04
Validation auc = 0.705634
Epoch 150
Validation auc = 0.702347
Epoch 151
Validation auc = 0.703756
Epoch 152
Validation auc = 0.699531
Epoch 153
Validation auc = 0.697653
Epoch 154
Loss = 1.1672e-02, PNorm = 67.7754, GNorm = 0.3165, lr_0 = 7.7759e-04
Validation auc = 0.690141
Epoch 155
Validation auc = 0.687324
Epoch 156
Validation auc = 0.687324
Epoch 157
Validation auc = 0.685915
Epoch 158
Validation auc = 0.685446
Epoch 159
Loss = 1.7518e-02, PNorm = 67.9069, GNorm = 0.4031, lr_0 = 7.7095e-04
Validation auc = 0.685446
Epoch 160
Validation auc = 0.684507
Epoch 161
Validation auc = 0.683568
Epoch 162
Validation auc = 0.687793
Epoch 163
Validation auc = 0.693897
Epoch 164
Loss = 2.5640e-02, PNorm = 68.0118, GNorm = 0.7343, lr_0 = 7.6436e-04
Validation auc = 0.694366
Epoch 165
Validation auc = 0.693427
Epoch 166
Validation auc = 0.689202
Epoch 167
Validation auc = 0.688732
Epoch 168
Validation auc = 0.691080
Epoch 169
Loss = 1.6053e-02, PNorm = 68.1129, GNorm = 0.3615, lr_0 = 7.5782e-04
Validation auc = 0.689671
Epoch 170
Validation auc = 0.686854
Epoch 171
Validation auc = 0.684507
Epoch 172
Validation auc = 0.684507
Epoch 173
Validation auc = 0.686385
Epoch 174
Loss = 3.3259e-02, PNorm = 68.2237, GNorm = 0.1290, lr_0 = 7.5134e-04
Validation auc = 0.691080
Epoch 175
Validation auc = 0.697183
Epoch 176
Validation auc = 0.703286
Epoch 177
Validation auc = 0.704695
Epoch 178
Validation auc = 0.704695
Epoch 179
Loss = 1.6168e-02, PNorm = 68.3485, GNorm = 0.3699, lr_0 = 7.4492e-04
Validation auc = 0.700000
Epoch 180
Validation auc = 0.697183
Epoch 181
Validation auc = 0.695305
Epoch 182
Validation auc = 0.693427
Epoch 183
Validation auc = 0.690610
Epoch 184
Loss = 3.6251e-02, PNorm = 68.4751, GNorm = 1.2238, lr_0 = 7.3855e-04
Validation auc = 0.684977
Epoch 185
Validation auc = 0.674648
Epoch 186
Validation auc = 0.661033
Epoch 187
Validation auc = 0.661033
Epoch 188
Validation auc = 0.662911
Epoch 189
Loss = 8.7548e-03, PNorm = 68.6086, GNorm = 0.2900, lr_0 = 7.3224e-04
Validation auc = 0.666197
Epoch 190
Validation auc = 0.671831
Epoch 191
Validation auc = 0.671362
Epoch 192
Validation auc = 0.669484
Epoch 193
Validation auc = 0.668545
Epoch 194
Loss = 3.3868e-02, PNorm = 68.7035, GNorm = 0.8594, lr_0 = 7.2598e-04
Validation auc = 0.670892
Epoch 195
Validation auc = 0.671362
Epoch 196
Validation auc = 0.671831
Epoch 197
Validation auc = 0.675587
Epoch 198
Validation auc = 0.677934
Epoch 199
Loss = 1.0625e-02, PNorm = 68.8117, GNorm = 0.1369, lr_0 = 7.1977e-04
Validation auc = 0.681221
Epoch 200
Validation auc = 0.684977
Epoch 201
Validation auc = 0.685915
Epoch 202
Validation auc = 0.684038
Epoch 203
Validation auc = 0.684507
Epoch 204
Loss = 6.8962e-03, PNorm = 68.8955, GNorm = 0.1605, lr_0 = 7.1362e-04
Validation auc = 0.687324
Epoch 205
Validation auc = 0.689671
Epoch 206
Validation auc = 0.692019
Epoch 207
Validation auc = 0.690141
Epoch 208
Validation auc = 0.690141
Epoch 209
Loss = 3.4516e-03, PNorm = 68.9650, GNorm = 0.0593, lr_0 = 7.0752e-04
Validation auc = 0.694836
Epoch 210
Validation auc = 0.697653
Epoch 211
Validation auc = 0.698122
Epoch 212
Validation auc = 0.687324
Epoch 213
Validation auc = 0.679343
Epoch 214
Loss = 1.9182e-02, PNorm = 69.0258, GNorm = 0.6825, lr_0 = 7.0147e-04
Validation auc = 0.678873
Epoch 215
Validation auc = 0.676056
Epoch 216
Validation auc = 0.676526
Epoch 217
Validation auc = 0.678404
Epoch 218
Validation auc = 0.683099
Epoch 219
Loss = 1.4460e-02, PNorm = 69.0932, GNorm = 0.0981, lr_0 = 6.9548e-04
Validation auc = 0.690610
Epoch 220
Validation auc = 0.697653
Epoch 221
Validation auc = 0.702817
Epoch 222
Validation auc = 0.708451
Epoch 223
Validation auc = 0.717840
Epoch 224
Loss = 1.1837e-02, PNorm = 69.1526, GNorm = 0.0952, lr_0 = 6.8953e-04
Validation auc = 0.716901
Epoch 225
Validation auc = 0.719249
Epoch 226
Validation auc = 0.708451
Epoch 227
Validation auc = 0.700469
Epoch 228
Validation auc = 0.699061
Epoch 229
Loss = 7.3515e-03, PNorm = 69.2175, GNorm = 0.1566, lr_0 = 6.8364e-04
Validation auc = 0.699061
Epoch 230
Validation auc = 0.696714
Epoch 231
Validation auc = 0.691549
Epoch 232
Validation auc = 0.681221
Epoch 233
Validation auc = 0.677465
Epoch 234
Loss = 3.3410e-02, PNorm = 69.2907, GNorm = 1.2510, lr_0 = 6.7779e-04
Validation auc = 0.676056
Epoch 235
Validation auc = 0.676056
Epoch 236
Validation auc = 0.675587
Epoch 237
Validation auc = 0.674178
Epoch 238
Validation auc = 0.678873
Epoch 239
Loss = 7.7376e-03, PNorm = 69.3629, GNorm = 0.2574, lr_0 = 6.7200e-04
Validation auc = 0.681690
Epoch 240
Validation auc = 0.682629
Epoch 241
Validation auc = 0.687793
Epoch 242
Validation auc = 0.688732
Epoch 243
Validation auc = 0.686854
Epoch 244
Loss = 1.3179e-02, PNorm = 69.4254, GNorm = 0.0915, lr_0 = 6.6625e-04
Validation auc = 0.687793
Epoch 245
Validation auc = 0.692019
Epoch 246
Validation auc = 0.693427
Epoch 247
Validation auc = 0.698592
Epoch 248
Validation auc = 0.698122
Epoch 249
Loss = 9.3680e-03, PNorm = 69.4967, GNorm = 0.1261, lr_0 = 6.6056e-04
Validation auc = 0.694366
Epoch 250
Validation auc = 0.694366
Epoch 251
Validation auc = 0.699061
Epoch 252
Validation auc = 0.700939
Epoch 253
Validation auc = 0.699531
Epoch 254
Loss = 4.1832e-03, PNorm = 69.5678, GNorm = 0.0549, lr_0 = 6.5491e-04
Validation auc = 0.704695
Epoch 255
Validation auc = 0.703286
Epoch 256
Validation auc = 0.702817
Epoch 257
Validation auc = 0.700939
Epoch 258
Validation auc = 0.698592
Epoch 259
Loss = 6.7651e-03, PNorm = 69.6222, GNorm = 0.2128, lr_0 = 6.4931e-04
Validation auc = 0.696244
Epoch 260
Validation auc = 0.692488
Epoch 261
Validation auc = 0.691549
Epoch 262
Validation auc = 0.690141
Epoch 263
Validation auc = 0.688732
Epoch 264
Loss = 1.8853e-02, PNorm = 69.6761, GNorm = 0.1997, lr_0 = 6.4376e-04
Validation auc = 0.686854
Epoch 265
Validation auc = 0.685915
Epoch 266
Validation auc = 0.686854
Epoch 267
Validation auc = 0.685446
Epoch 268
Validation auc = 0.684977
Epoch 269
Loss = 1.5980e-02, PNorm = 69.7461, GNorm = 0.1769, lr_0 = 6.3826e-04
Validation auc = 0.687324
Epoch 270
Validation auc = 0.685915
Epoch 271
Validation auc = 0.684507
Epoch 272
Validation auc = 0.684038
Epoch 273
Validation auc = 0.682629
Epoch 274
Loss = 4.0218e-03, PNorm = 69.8216, GNorm = 0.1276, lr_0 = 6.3280e-04
Validation auc = 0.680282
Epoch 275
Validation auc = 0.679812
Epoch 276
Validation auc = 0.679812
Epoch 277
Validation auc = 0.678873
Epoch 278
Validation auc = 0.676995
Epoch 279
Loss = 1.6118e-03, PNorm = 69.8858, GNorm = 0.0211, lr_0 = 6.2739e-04
Validation auc = 0.675587
Epoch 280
Validation auc = 0.675117
Epoch 281
Validation auc = 0.671831
Epoch 282
Validation auc = 0.671362
Epoch 283
Validation auc = 0.672300
Epoch 284
Loss = 1.3054e-03, PNorm = 69.9410, GNorm = 0.0257, lr_0 = 6.2203e-04
Validation auc = 0.674178
Epoch 285
Validation auc = 0.677465
Epoch 286
Validation auc = 0.675587
Epoch 287
Validation auc = 0.676526
Epoch 288
Validation auc = 0.674178
Epoch 289
Loss = 8.9162e-04, PNorm = 69.9825, GNorm = 0.0484, lr_0 = 6.1671e-04
Validation auc = 0.676056
Epoch 290
Validation auc = 0.672770
Epoch 291
Validation auc = 0.673709
Epoch 292
Validation auc = 0.674648
Epoch 293
Validation auc = 0.672770
Epoch 294
Loss = 1.0001e-02, PNorm = 70.0166, GNorm = 0.0052, lr_0 = 6.1144e-04
Validation auc = 0.667606
Epoch 295
Validation auc = 0.667136
Epoch 296
Validation auc = 0.668075
Epoch 297
Validation auc = 0.665728
Epoch 298
Validation auc = 0.664789
Epoch 299
Loss = 6.8367e-03, PNorm = 70.0561, GNorm = 0.3244, lr_0 = 6.0621e-04
Validation auc = 0.670423
Model 0 best validation auc = 0.758216 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.934979
Ensemble test auc = 0.934979
10-fold cross validation
	Seed 0 ==> test auc = 0.898254
	Seed 1 ==> test auc = 0.921734
	Seed 2 ==> test auc = 0.873570
	Seed 3 ==> test auc = 0.875978
	Seed 4 ==> test auc = 0.919326
	Seed 5 ==> test auc = 0.904877
	Seed 6 ==> test auc = 0.848886
	Seed 7 ==> test auc = 0.852498
	Seed 8 ==> test auc = 0.889223
	Seed 9 ==> test auc = 0.934979
Overall test auc = 0.891933 +/- 0.027760
Elapsed time = 0:09:48
